DL QuickStart

来自Jack's Lab
（版本间的差异）
跳转到： 导航, 搜索
(→‎Computor Vision)
(→‎Notes)
 
（未显示1个用户的82个中间版本）
第1行： 第1行：
 
== Overview ==
 
== Overview ==
 +
 +
* Convolutional Neural Networks，CNN，卷积神经网络
 +
* Recurrent Neural Networks, RNN，循环神经网络
  
 
* Deep Belief Networks 深度信念网络
 
* Deep Belief Networks 深度信念网络
第6行： 第9行：
  
  
−
* Convolutional Neural Networks，CNN，卷积神经网络
+
CNN 目前是图像识别（计算机视觉）最有效也是使用最多的
−
* Recurrent Neural Networks, RNN，循环神经网络
+
  
−
 
+
RNN 则多用于自然语言处理、语音识别、手写体识别等
−
 
+
−
CNN 目前是图像识别最有效也是使用最多的
+
−
 
+
−
自然语言处理、语音识别、手写体识别则多用 RNN
+
  
  
 
https://www.zhihu.com/question/31785984/answer/55063559
 
https://www.zhihu.com/question/31785984/answer/55063559
  
−
<br><br>
 
  
−
== RNN ==
+
'''Gitbook:'''
  
−
http://blog.csdn.net/heyongluoyao8/article/details/48636251
+
* [https://xhhjin.gitbooks.io/neural-networks-and-deep-learning-zh/content/ 神经网络与深度学习] Michael Nielsen
 +
* [https://www.gitbook.com/book/hit-scir/neural-networks-and-deep-learning-zh_cn/details 神经网络与深度学习2]
  
−
http://rayz0620.github.io/2015/05/14/rnn_note_1/
 
  
−
https://www.zhihu.com/question/34681168
+
* [https://www.pyimagesearch.com/ The Deep Learning for Computer Vision with Python]
  
−
<br><br>
 
  
−
== Framework ==
+
'''Blog:'''
  
−
推荐斯坦福的 CS231n 课程
+
* [http://www.pyimagesearch.com/start-here-learn-computer-vision-opencv/ Deep Learning for Computer Vision with Python]
 +
* [http://www.pyimagesearch.com/2016/08/01/lenet-convolutional-neural-network-in-python/ LeNet – Convolutional Neural Network in Python]
  
−
在 Lecture 12 中，JJ 对于几个主流的库的应用情况，优缺点逐个做了详细的说明：
 
  
−
 
+
* [http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html Image Classification Datasets Results]
−
[[文件:Dl-framwork-compare.png]]
+
  
 
<br><br>
 
<br><br>
第75行： 第70行：
  
 
* [https://zhuanlan.zhihu.com/p/22107715 循环神经网络惊人的有效性（上）]
 
* [https://zhuanlan.zhihu.com/p/22107715 循环神经网络惊人的有效性（上）]
 +
 +
<br><br>
 +
 +
== CNN ==
 +
 +
* [https://arxiv.org/pdf/1311.2901.pdf Visualizing and Understanding Convolutional Networks]
 +
 +
* https://docs.ultralytics.com/zh/models/yolo12/
 +
 +
=== Object Detection ===
 +
 +
* SSD_MobileNet_v1  300x300 (1055fps)
 +
* YOLOv5m    640x640 (218fps)
 +
** YOLOv4_tiny (610fps)
 +
** YOLOv6n    (345fps)
 +
** YOLOv7      (45fps)
 +
** YOLOx_s_wide (75fps)
 +
** YOLOv3      (26fps)
 +
** YOLOv8n    (270fps)
 +
** YOLOv8s    (128fps)
 +
** YOLOv8m    (55fps)
 +
 +
<br>
 +
 +
=== Segmentation ===
 +
* stdc1  1024x1920 34fps
 +
 +
<br>
 +
 +
=== Multi stream object detection (8stream)===
 +
* YOLOv3    608x608 66fps
 +
 +
<br>
 +
 +
=== Classification ===
 +
* ResNet-50v1          224x224, 1331fps
 +
* MobileNet_v2_1.0    224x224, 2444fps
 +
* EfficientNet_M      240x240, 890fps
 +
 +
<br>
 +
 +
== RNN ==
 +
 +
http://blog.csdn.net/heyongluoyao8/article/details/48636251
 +
 +
http://rayz0620.github.io/2015/05/14/rnn_note_1/
 +
 +
https://www.zhihu.com/question/34681168
 +
 +
<br><br>
 +
 +
== Framework ==
 +
 +
推荐斯坦福的 CS231n 课程
 +
 +
在 Lecture 12 中，JJ 对于几个主流的库的应用情况，优缺点逐个做了详细的说明：
 +
 +
 +
[[文件:Dl-framwork-compare.png]]
 +
 +
<br>
 +
 +
=== TensorFlow ===
 +
 +
* [https://docs.scipy.org/doc/numpy-dev/user/quickstart.html NumPy Quick Start]
 +
* [http://www.tensorfly.cn/tfdoc/get_started/introduction.html TensorFlow Quick Start CN]
 +
* [https://www.tensorflow.org/get_started/mnist/pros TensorFlow Quick Start]
 +
* [https://medium.com/@hamedmp/exporting-trained-tensorflow-models-to-c-the-right-way-cf24b609d183 Export TensorFlow Model to C++]
 +
 +
 +
==== Install ====
 +
 +
<source lang=bash>
 +
$ sudo apt-get install python-pip python-dev  # for Python 2.7
 +
$ sudo apt-get install python3-pip python3-dev # for Python 3.n
 +
 +
$ pip install tensorflow      # Python 2.7; CPU support (no GPU support)
 +
$ pip3 install tensorflow    # Python 3.n; CPU support (no GPU support)
 +
$ pip install tensorflow-gpu  # Python 2.7;  GPU support
 +
$ pip3 install tensorflow-gpu # Python 3.n; GPU support
 +
</source>
 +
 +
<source lang=bash>
 +
$ pip install -i https://pypi.tuna.tsinghua.edu.cn/simple tensorflow-gpu==2.5.0
 +
</source>
 +
 +
<br><br>
 +
 +
==== Uninstall ====
 +
 +
<source lang=bash>
 +
$ sudo pip uninstall tensorflow  # for Python 2.7
 +
$ sudo pip3 uninstall tensorflow # for Python 3.n
 +
</source>
 +
 +
<br><br>
 +
 +
==== Build from Git in Windows ====
 +
 +
The latest guide: https://tensorflow.google.cn/install/source_windows
 +
 +
<br><br>
 +
 +
==== Build from Git in Linux ====
 +
 +
https://tensorflow.google.cn/install/source
 +
 +
<source lang=bash>
 +
$ git clone https://github.com/tensorflow/tensorflow
 +
$ cd tensorflow
 +
$ git checkout r1.0
 +
</source>
 +
 +
* Install Bazel: https://bazel.build/versions/master/docs/install.html
 +
** https://docs.bazel.build/versions/main/install-ubuntu.html
 +
** https://github.com/bazelbuild/bazel/releases?expanded=true&page=2&q=0.25
 +
 +
<source lang=bash>
 +
$ sudo apt-get install python-numpy python-dev python-pip python-wheel
 +
 +
$ sudo apt-get install python3-numpy python3-dev python3-pip python3-wheel
 +
 +
$ sudo apt-get install libcupti-dev
 +
</source>
 +
 +
 +
<source lang=bash>
 +
$ cd tensorflow  # cd to the top-level directory created
 +
$ ./configure
 +
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python2.7
 +
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
 +
Do you wish to use jemalloc as the malloc implementation? [Y/n]
 +
jemalloc enabled
 +
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
 +
No Google Cloud Platform support will be enabled for TensorFlow
 +
Do you wish to build TensorFlow with Hadoop File System support? [y/N]
 +
No Hadoop File System support will be enabled for TensorFlow
 +
Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
 +
No XLA JIT support will be enabled for TensorFlow
 +
Found possible Python library paths:
 +
  /usr/local/lib/python2.7/dist-packages
 +
  /usr/lib/python2.7/dist-packages
 +
Please input the desired Python library path to use.  Default is [/usr/local/lib/python2.7/dist-packages]
 +
Using python library path: /usr/local/lib/python2.7/dist-packages
 +
Do you wish to build TensorFlow with OpenCL support? [y/N] N
 +
No OpenCL support will be enabled for TensorFlow
 +
Do you wish to build TensorFlow with CUDA support? [y/N] Y
 +
CUDA support will be enabled for TensorFlow
 +
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
 +
Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 8.0
 +
Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
 +
Please specify the cuDNN version you want to use. [Leave empty to use system default]: 5
 +
Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
 +
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
 +
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
 +
Please note that each additional compute capability significantly increases your build time and binary size.
 +
[Default is: "3.5,5.2"]: 3.0
 +
Setting up Cuda include
 +
Setting up Cuda lib
 +
Setting up Cuda bin
 +
Setting up Cuda nvvm
 +
Setting up CUPTI include
 +
Setting up CUPTI lib64
 +
Configuration finished
 +
 +
$ bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
 +
 +
$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
 +
 +
$ sudo pip install /tmp/tensorflow_pkg/tensorflow-1.2.0-py2-none-any.whl
 +
</source>
 +
 +
<br><br>
 +
 +
==== Validate your installation ====
 +
 +
<source lang=bash>
 +
$ python
 +
# Python
 +
import tensorflow as tf
 +
hello = tf.constant('Hello, TensorFlow!')
 +
sess = tf.Session()
 +
print(sess.run(hello))
 +
 +
Hello, TensorFlow!
 +
</source>
 +
 +
<br><br>
 +
 +
=== DarkNet ===
 +
 +
* [https://pjreddie.com/darknet/tiny-darknet/ Tiny DarkNet]
 +
 +
<br><br>
 +
 +
=== Caffe ===
 +
 +
* [http://hanzratech.in/2015/07/27/installing-caffe-on-ubuntu.html Caffe Installation on Ubuntu 14.04 (CPU) with PYTHON support]
 +
 +
<br><br>
 +
 +
== Hardware ==
 +
 +
=== GPU Architecture ===
 +
 +
* [http://haifux.org/lectures/267/Introduction-to-GPUs.pdf Introduction to GPU architecture]
 +
* [[Nvidia GPU Architecture]]
 +
 +
<br><br>
 +
 +
=== TPU Architecture ===
 +
 +
Tensor Processing Unit(or TPUs) are application-specific integrated circuits (ASICs) developed specifically for machine learning.
 +
 +
* [https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk/view Paper]
 +
* [https://cloudplatform.googleblog.com/2017/04/quantifying-the-performance-of-the-TPU-our-first-machine-learning-chip.html Blog] Quantifying the performance of the TPU, our first machine learning chip 2017-04-05
 +
 +
<br><br>
 +
 +
== Notes ==
 +
 +
=== Nvidia drivers in windows ===
 +
 +
安装完驱动，就会有 nvidia-smi 这个工具：
 +
 +
<source lang=bash>
 +
DELL@BH1RBH MINGW64 ~
 +
$ nvidia-smi
 +
Fri Feb 28 10:10:24 2025
 +
+-----------------------------------------------------------------------------------------+
 +
| NVIDIA-SMI 560.94                Driver Version: 560.94        CUDA Version: 12.6    |
 +
|-----------------------------------------+------------------------+----------------------+
 +
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
 +
| Fan  Temp  Perf          Pwr:Usage/Cap |          Memory-Usage | GPU-Util  Compute M. |
 +
|                                        |                        |              MIG M. |
 +
|=========================================+========================+======================|
 +
|  0  NVIDIA GeForce GTX 980      WDDM  |  00000000:01:00.0 Off |                  N/A |
 +
| 33%  56C    P0            49W /  180W |      0MiB /  4096MiB |      0%      Default |
 +
|                                        |                        |                  N/A |
 +
+-----------------------------------------+------------------------+----------------------+
 +
 +
+-----------------------------------------------------------------------------------------+
 +
| Processes:                                                                              |
 +
|  GPU  GI  CI        PID  Type  Process name                              GPU Memory |
 +
|        ID  ID                                                              Usage      |
 +
|=========================================================================================|
 +
|    0  N/A  N/A    11688      C  ...win-pre1.0\system\python\python.exe      N/A      |
 +
+-----------------------------------------------------------------------------------------+
 +
</source>
 +
 +
<br>
 +
 +
=== Install Nvidia driver for ubuntu 16.04 ===
 +
 +
Blacklist the modules. Open the blacklist.conf file.
 +
 +
add the following modules in the file /etc/modprobe.d/blacklist.conf
 +
 +
<pre>
 +
#this might not be required for x86 32 bit users
 +
blacklist amd76x_edac
 +
blacklist vga16fb
 +
blacklist nouveau
 +
blacklist rivafb
 +
blacklist nvidiafb
 +
blacklist rivatv
 +
</pre>
 +
 +
 +
<source lang=bash>
 +
$ sudo update-initramfs -u
 +
</source>
 +
 +
 +
reboot the system, then:
 +
 +
<source lang=bash>
 +
$ sudo apt-get install gcc libc-dev make
 +
$ sudo ./NVIDIA-Linux-x86_64-343.22.run # for GeForce 980
 +
 +
$ sudo ./NVIDIA-Linux-x86_64-381.22.run # for GeForce 1050M
 +
</source>
 +
 +
<br><br>
 +
 +
=== Install CUDA Toolkit ===
 +
 +
Download: https://developer.nvidia.com/cuda-downloads
 +
 +
<source lang=bash>
 +
$ sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb
 +
$ sudo apt-get update
 +
$ sudo apt-get install cuda
 +
</source>
 +
 +
add the following lines into /etc/bash.bashrc:
 +
 +
<source lang=bash>
 +
export PATH=/usr/local/cuda/bin:$PATH
 +
export LD_LIBRARY_PATH="/usr/local/cuda/lib64 $LD_LIBRARY_PATH"
 +
</source>
  
 
<br><br>
 
<br><br>
第94行： 第390行：
 
* [http://thecodacus.com/face-recognition-opencv-train-recognizer/ Face Recognition OpenCV – Training A Face Recognizer]
 
* [http://thecodacus.com/face-recognition-opencv-train-recognizer/ Face Recognition OpenCV – Training A Face Recognizer]
 
* [http://thecodacus.com/face-recognition-loading-recognizer/ Face Recognition Using OpenCV | Loading Recognizer]
 
* [http://thecodacus.com/face-recognition-loading-recognizer/ Face Recognition Using OpenCV | Loading Recognizer]
 +
* https://zhuanlan.zhihu.com/p/25025596
 +
 +
 +
* [http://www.ebaina.com/bbs/forum.php?mod=viewthread&tid=4858 海思移植opencv+人脸检测]
 +
* http://shervinemami.info/faceRecognition.html
 +
 +
* [http://ahogrammer.com/2016/11/15/deep-learning-enables-you-to-hide-screen-when-your-boss-is-approaching/ Boss Sensor]
 +
* [https://medium.com/@hamedmp/exporting-trained-tensorflow-models-to-c-the-right-way-cf24b609d183 Exporting trained TensorFlow models to C++]
  
  
第101行： 第405行：
 
* [http://eyalarubas.com/face-detection-and-recognition.html Face Detection and Recognition (Theory and Practice)]
 
* [http://eyalarubas.com/face-detection-and-recognition.html Face Detection and Recognition (Theory and Practice)]
  
 +
 +
* [https://zhuanlan.zhihu.com/p/24816781 基于深度学习的人脸识别技术综述]
 +
* [https://zhuanlan.zhihu.com/p/24837264 谷歌人脸识别系统FaceNet解析]
 +
* [https://zhuanlan.zhihu.com/p/25025596 基于mtcnn和facenet的实时人脸检测与识别系统开发]
 +
* [https://zhuanlan.zhihu.com/p/24316690 使用深度卷积神经网络方法进行3D人脸重建]
 +
 +
 +
* [http://blog.csdn.net/cv_family_z/article/details/51917143 CNN网络二值化--XNOR-Net]
 +
 +
* RAISR（Rapid and Accurate Image Super-Resolution），利用机器学习将低分辨率图像转化为高分辨率图像。这项技术能够在节省带宽75%的情况下分辨率效果达到甚至超过原图，同时速度能够提升大约10到100倍
 +
 +
<br><br>
 +
 +
==== OpenFace ====
 +
 +
* http://cmusatyalab.github.io/openface/
 +
* https://github.com/cmusatyalab/openface
 +
 +
* http://shamangary.logdown.com/posts/800267-openface-installation
 +
 +
 +
* [https://github.com/BrandonJoffe/home_surveillance Home surveillance dashboard that uses OpenFace for face recognition]
 +
* [https://github.com/pyannote/pyannote-video Face detection, tracking, and clustering in videos using OpenFace] --- [http://nbviewer.jupyter.org/github/pyannote/pyannote-video/blob/master/doc/getting_started.ipynb Getting Start]
 +
* [https://github.com/davidsandberg/facenet Face recognition using Tensorflow]
 +
* [https://github.com/aybassiouny/OpenFaceCpp C++ implementation for OpenFace library by CMU]
 +
 +
<br><br>
 +
 +
==== Facial Keypoints Detection ====
 +
 +
* https://www.kaggle.com/c/facial-keypoints-detection Facial Keypoints Detection - Kaggle project
 +
* [https://github.com/dnouri/kfkd-tutorial dnouri's Kaggle Facial Keypoints Detection tutorial]
 +
* https://github.com/saber1988/facial-keypoints-detection
 +
 +
<br><br>
 +
 +
==== Facial Emotion Recognition ====
 +
 +
* [https://github.com/JostineHo/mememoji A facial expression classification system that recognizes 6 basic emotions: happy, sad, surprise, fear, anger and neutral]
 +
* [http://cs231n.stanford.edu/reports2016/022_Report.pdf Facial Emotion Recognition in Real Time]
 +
* [https://arxiv.org/pdf/1408.3750.pdf Real-time emotion recognition for gaming using deep convolutional network features]
 +
 +
 +
<br><br>
 +
 +
==== Texture Matching ====
 +
 +
* [http://hanzratech.in/2015/05/30/local-binary-patterns.html Texture Matching using Local Binary Patterns (LBP), OpenCV, scikit-learn and Python]
  
 
<br><br>
 
<br><br>
第146行： 第498行：
 
*http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.242.9890
 
*http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.242.9890
  
−
<br><br>
+
 
−
<br><br>
+
* [https://zhuanlan.zhihu.com/p/23255679 寒武纪——一个神经网络指令集架构]
−
<br><br>
+
* [https://tech.meituan.com/O2O_Intelligent_distribution.html 即时配送的订单分配策略：从建模和优化]
−
<br><br>
+
 
−
<br><br>
+
* [[Financial]]
−
<br><br>
+
* [http://www.meng.uno/articles/5b828fc2/ 机器学习&深度学习速查表]
−
<br><br>
+
* [https://www.leiphone.com/news/201707/TmZfbVdmupizoVcB.html 深度神经网络可视化工具集锦]
−
<br><br>
+
* [https://blog.csdn.net/han2529386161/article/details/102723482 关于 MLPerf 的一些调查]
−
<br><br>
+
 
−
<br><br>
+
<br>
−
<br><br>
+
−
<br><br>
+
−
<br><br>
+
−
<br><br>
+
−
<br><br>
+
−
<br><br>
+
−
<br><br>
+

2025年2月28日 (五) 10:12的最后版本

目录

[编辑] 1 Overview

  • Convolutional Neural Networks，CNN，卷积神经网络
  • Recurrent Neural Networks, RNN，循环神经网络
  • Deep Belief Networks 深度信念网络
  • Restricted Boltzmann Machine 受限波尔兹曼机
  • Stacked Auto-encoders 堆栈式自动编码器


CNN 目前是图像识别（计算机视觉）最有效也是使用最多的

RNN 则多用于自然语言处理、语音识别、手写体识别等


https://www.zhihu.com/question/31785984/answer/55063559


Gitbook:



Blog:




[编辑] 2 CS231n

知乎智能单元： https://zhuanlan.zhihu.com/intelligentunit


CS231n官方笔记授权翻译总集篇发布







[编辑] 3 CNN

[编辑] 3.1 Object Detection

  • SSD_MobileNet_v1 300x300 (1055fps)
  • YOLOv5m 640x640 (218fps)
    • YOLOv4_tiny (610fps)
    • YOLOv6n (345fps)
    • YOLOv7 (45fps)
    • YOLOx_s_wide (75fps)
    • YOLOv3 (26fps)
    • YOLOv8n (270fps)
    • YOLOv8s (128fps)
    • YOLOv8m (55fps)


[编辑] 3.2 Segmentation

  • stdc1 1024x1920 34fps


[编辑] 3.3 Multi stream object detection (8stream)

  • YOLOv3 608x608 66fps


[编辑] 3.4 Classification

  • ResNet-50v1 224x224, 1331fps
  • MobileNet_v2_1.0 224x224, 2444fps
  • EfficientNet_M 240x240, 890fps


[编辑] 4 RNN

http://blog.csdn.net/heyongluoyao8/article/details/48636251

http://rayz0620.github.io/2015/05/14/rnn_note_1/

https://www.zhihu.com/question/34681168



[编辑] 5 Framework

推荐斯坦福的 CS231n 课程

在 Lecture 12 中，JJ 对于几个主流的库的应用情况，优缺点逐个做了详细的说明：


Dl-framwork-compare.png


[编辑] 5.1 TensorFlow


[编辑] 5.1.1 Install 

$ sudo apt-get install python-pip python-dev   # for Python 2.7
$ sudo apt-get install python3-pip python3-dev # for Python 3.n

 $ pip install tensorflow      # Python 2.7; CPU support (no GPU support)
 $ pip3 install tensorflow     # Python 3.n; CPU support (no GPU support)
 $ pip install tensorflow-gpu  # Python 2.7;  GPU support
 $ pip3 install tensorflow-gpu # Python 3.n; GPU support 

$ pip install -i https://pypi.tuna.tsinghua.edu.cn/simple tensorflow-gpu==2.5.0



[编辑] 5.1.2 Uninstall 

$ sudo pip uninstall tensorflow  # for Python 2.7
$ sudo pip3 uninstall tensorflow # for Python 3.n



[编辑] 5.1.3 Build from Git in Windows

The latest guide: https://tensorflow.google.cn/install/source_windows



[编辑] 5.1.4 Build from Git in Linux

https://tensorflow.google.cn/install/source 

$ git clone https://github.com/tensorflow/tensorflow 
$ cd tensorflow
$ git checkout r1.0

$ sudo apt-get install python-numpy python-dev python-pip python-wheel

$ sudo apt-get install python3-numpy python3-dev python3-pip python3-wheel

 $ sudo apt-get install libcupti-dev 



$ cd tensorflow  # cd to the top-level directory created
$ ./configure
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python2.7
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
Do you wish to use jemalloc as the malloc implementation? [Y/n]
jemalloc enabled
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N]
No Hadoop File System support will be enabled for TensorFlow
Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
No XLA JIT support will be enabled for TensorFlow
Found possible Python library paths:
  /usr/local/lib/python2.7/dist-packages
  /usr/lib/python2.7/dist-packages
Please input the desired Python library path to use.  Default is [/usr/local/lib/python2.7/dist-packages]
Using python library path: /usr/local/lib/python2.7/dist-packages
Do you wish to build TensorFlow with OpenCL support? [y/N] N
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with CUDA support? [y/N] Y
CUDA support will be enabled for TensorFlow
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 8.0
Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify the cuDNN version you want to use. [Leave empty to use system default]: 5
Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size.
[Default is: "3.5,5.2"]: 3.0
Setting up Cuda include
Setting up Cuda lib
Setting up Cuda bin
Setting up Cuda nvvm
Setting up CUPTI include
Setting up CUPTI lib64
Configuration finished

$ bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

$ sudo pip install /tmp/tensorflow_pkg/tensorflow-1.2.0-py2-none-any.whl



[编辑] 5.1.5 Validate your installation 

$ python
# Python
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))

Hello, TensorFlow!



[编辑] 5.2 DarkNet



[编辑] 5.3 Caffe



[编辑] 6 Hardware

[编辑] 6.1 GPU Architecture



[编辑] 6.2 TPU Architecture

Tensor Processing Unit(or TPUs) are application-specific integrated circuits (ASICs) developed specifically for machine learning.

  • Paper
  • Blog Quantifying the performance of the TPU, our first machine learning chip 2017-04-05



[编辑] 7 Notes

[编辑] 7.1 Nvidia drivers in windows

安装完驱动，就会有 nvidia-smi 这个工具： 

DELL@BH1RBH MINGW64 ~
$ nvidia-smi
Fri Feb 28 10:10:24 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.94                 Driver Version: 560.94         CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 980       WDDM  |   00000000:01:00.0 Off |                  N/A |
| 33%   56C    P0             49W /  180W |       0MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A     11688      C   ...win-pre1.0\system\python\python.exe      N/A      |
+-----------------------------------------------------------------------------------------+


[编辑] 7.2 Install Nvidia driver for ubuntu 16.04

Blacklist the modules. Open the blacklist.conf file.

add the following modules in the file /etc/modprobe.d/blacklist.conf 

#this might not be required for x86 32 bit users
blacklist amd76x_edac
blacklist vga16fb
blacklist nouveau
blacklist rivafb
blacklist nvidiafb
blacklist rivatv



$ sudo update-initramfs -u


reboot the system, then: 

$ sudo apt-get install gcc libc-dev make 
$ sudo ./NVIDIA-Linux-x86_64-343.22.run # for GeForce 980

$ sudo ./NVIDIA-Linux-x86_64-381.22.run # for GeForce 1050M



[编辑] 7.3 Install CUDA Toolkit

Download: https://developer.nvidia.com/cuda-downloads 

$ sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb
$ sudo apt-get update
$ sudo apt-get install cuda

add the following lines into /etc/bash.bashrc: 

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH="/usr/local/cuda/lib64 $LD_LIBRARY_PATH"



[编辑] 8 OpenAI

从OpenAI看深度学习研究前沿



[编辑] 9 Applications

[编辑] 9.1 Computor Vision

[编辑] 9.1.1 Face detection





  • RAISR（Rapid and Accurate Image Super-Resolution），利用机器学习将低分辨率图像转化为高分辨率图像。这项技术能够在节省带宽75%的情况下分辨率效果达到甚至超过原图，同时速度能够提升大约10到100倍



[编辑] 9.1.2 OpenFace




[编辑] 9.1.3 Facial Keypoints Detection



[编辑] 9.1.4 Facial Emotion Recognition




[编辑] 9.1.5 Texture Matching



[编辑] 10 Furture

Deep Reinforcement Learning 深度增强学习资源 (持续更新）https://zhuanlan.zhihu.com/p/20885568?refer=intelligentunit

获取人工智能AI前沿信息 https://zhuanlan.zhihu.com/p/21263408?refer=intelligentunit




[编辑] 11 Reference

问：您对Jeff Hawkins对深度学习的批评有什么看法？Hawkins是On Intelligence一书的作者, 该书2004年出版，内容关于大脑如何工作，以及如何参考大脑来制造智能机器。他声称深度学习没有对时间序列建模。人脑是基于一系列的传感数据进行思考的，人的学习主要在于对序列模式的记忆，比如你看到一个搞怪猫的视频，实际是猫的动作让你发笑，而不是像Google公司所用的静态图片。参见这个链接

答：时间相关的神经网络其实有很多工作，递归神经网络模型对时间关系隐性建模，通常应用于语音识别。比如下面这两个工作。

[1] http://www.cs.toronto.edu/~hinton/absps/RNN13.pdf

[2] http://papers.nips.cc/paper/5166 ... neural-networks.pdf

还有这篇文章：http://arxiv.org/abs/1312.6026.

自然语言处理中的序列也有所考虑：http://arxiv.org/abs/1306.2795


问：根据我的理解，深度神经网络训练上的成功跟选取正确的超参数有关系，比如网络深度，隐含层的大小，稀疏约束值等等。有些论文基于随机搜索来寻找这些参数。可能跟代码写得好也有关系。有没有一个地方能让研究者找到某些特定任务的合理超参数呢？在这些参数的基础上，可能更容易找到更优化的参数。

答：可以看上文关于超参数的部分。James Bergstra 继续了这部分工作。我觉得有这么一个数据库，存储着许多推荐的超参数设置，对于神经网络训练是非常有好处的。Github上面的Hyperopt项目，做了类似的事情。hyperopt项目聚焦于神经网络、卷积网络，给出一些超参数设置的建议。以简单的因子分布的形式给出。比如隐含层的数量应该是1到3，每一层的隐含单元数目应该是50到5000。其实超参数还有很多，以及更好的超参数搜索算法等等。下面是更多的参考论文：


问：Bengio教授，在深度学习中，有那么一类方法，采用比较高级的数学如代数和拓扑集合。John Healy几年前声称通过通过范畴论（Category Theory）改进了神经网络（ART1）。您对于这类尝试有什么看法？是儿戏还是很有前途？

答：可以看看Morton和Montufar的工作，参考附加材料：

热带几何以及概率模型中的热带几何



来自“http://wiki.jackslab.org/index.php?title=DL_QuickStart&oldid=18107”
个人工具
名字空间

变换
操作
导航
工具箱