DL QuickStart
来自Jack's Lab
(版本间的差异)
(→Overview) |
(→Framework) |
||
第59行: | 第59行: | ||
* [http://hanzratech.in/2015/07/27/installing-caffe-on-ubuntu.html Caffe Installation on Ubuntu 14.04 (CPU) with PYTHON support] | * [http://hanzratech.in/2015/07/27/installing-caffe-on-ubuntu.html Caffe Installation on Ubuntu 14.04 (CPU) with PYTHON support] | ||
+ | |||
+ | <br><br> | ||
+ | |||
+ | == Notes == | ||
+ | |||
+ | === Install Nvidia driver for ubuntu 16.04 === | ||
+ | |||
+ | Blacklist the modules. Open the blacklist.conf file. | ||
+ | |||
+ | add the following modules in the file /etc/modprobe.d/blacklist.conf | ||
+ | |||
+ | <pre> | ||
+ | blacklist amd76x_edac #this might not be required for x86 32 bit users. | ||
+ | blacklist vga16fb | ||
+ | blacklist nouveau | ||
+ | blacklist rivafb | ||
+ | blacklist nvidiafb | ||
+ | blacklist rivatv | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | <source lang=bash> | ||
+ | $ sudo apt-get remove --purge nvidia* | ||
+ | $ sudo update-initramfs -u | ||
+ | </source> | ||
+ | |||
+ | |||
+ | reboot the system, then: | ||
+ | |||
+ | <source lang=bash> | ||
+ | $ sudo apt-get install gcc libc-dev | ||
+ | $ sudo ./NVIDIA-Linux-x86_64-343.22.run | ||
+ | <source> | ||
<br><br> | <br><br> |
2017年3月20日 (一) 15:37的版本
目录 |
1 Overview
- Deep Belief Networks 深度信念网络
- Restricted Boltzmann Machine 受限波尔兹曼机
- Stacked Auto-encoders 堆栈式自动编码器
- Convolutional Neural Networks,CNN,卷积神经网络
- Recurrent Neural Networks, RNN,循环神经网络
CNN 目前是图像识别最有效也是使用最多的
自然语言处理、语音识别、手写体识别则多用 RNN
https://www.zhihu.com/question/31785984/answer/55063559
Gitbook:
- 神经网络与深度学习 Michael Nielsen
- 神经网络与深度学习2
Blog:
2 RNN
http://blog.csdn.net/heyongluoyao8/article/details/48636251
http://rayz0620.github.io/2015/05/14/rnn_note_1/
https://www.zhihu.com/question/34681168
3 Framework
推荐斯坦福的 CS231n 课程
在 Lecture 12 中,JJ 对于几个主流的库的应用情况,优缺点逐个做了详细的说明:
3.1 Caffe
4 Notes
4.1 Install Nvidia driver for ubuntu 16.04
Blacklist the modules. Open the blacklist.conf file.
add the following modules in the file /etc/modprobe.d/blacklist.conf
blacklist amd76x_edac #this might not be required for x86 32 bit users. blacklist vga16fb blacklist nouveau blacklist rivafb blacklist nvidiafb blacklist rivatv
$ sudo apt-get remove --purge nvidia* $ sudo update-initramfs -u
reboot the system, then:
$ sudo apt-get install gcc libc-dev $ sudo ./NVIDIA-Linux-x86_64-343.22.run <source> <br><br> == CS231n == 知乎智能单元: https://zhuanlan.zhihu.com/intelligentunit [https://zhuanlan.zhihu.com/p/21930884 CS231n官方笔记授权翻译总集篇发布] * [https://zhuanlan.zhihu.com/p/21441838 简介] * [https://zhuanlan.zhihu.com/p/21407711 反向传播笔记]] * [https://zhuanlan.zhihu.com/p/21462488 神经网络笔记1(上)] * [https://zhuanlan.zhihu.com/p/21513367 神经网络笔记1(下)] * [https://zhuanlan.zhihu.com/p/21560667 神经网络笔记 2] * [https://zhuanlan.zhihu.com/p/21741716 神经网络笔记3(上)] * [https://zhuanlan.zhihu.com/p/21798784 神经网络笔记3(下)] * [https://zhuanlan.zhihu.com/p/22038289 卷积神经网络笔记] * [https://zhuanlan.zhihu.com/p/21360434 最优化笔记(上)] * [https://zhuanlan.zhihu.com/p/21387326 最优化笔记(下)] * [https://zhuanlan.zhihu.com/p/20918580 线性分类笔记(上)] * [https://zhuanlan.zhihu.com/p/20945670 线性分类笔记(中)] * [https://zhuanlan.zhihu.com/p/21102293 线性分类笔记(下)] * [https://zhuanlan.zhihu.com/p/20894041 图像分类笔记(上)] * [https://zhuanlan.zhihu.com/p/20900216 图像分类笔记(下)] * [https://zhuanlan.zhihu.com/p/20878530 Python Numpy教程] * [https://zhuanlan.zhihu.com/p/22107715 循环神经网络惊人的有效性(上)] <br><br> == OpenAI == [https://zhuanlan.zhihu.com/p/20924929 从OpenAI看深度学习研究前沿] <br><br> == Applications == === Computor Vision === ==== Face detection ==== * [http://thecodacus.com/opencv-python-face-detection/ Face Detection Using OpenCV In Python | How To Setup OpenCV Python] * [http://thecodacus.com/opencv-face-recognition-python-part1/ Face Recognition – OpenCV Python | Dataset Generator] * [http://thecodacus.com/face-recognition-opencv-train-recognizer/ Face Recognition OpenCV – Training A Face Recognizer] * [http://thecodacus.com/face-recognition-loading-recognizer/ Face Recognition Using OpenCV | Loading Recognizer] * https://zhuanlan.zhihu.com/p/25025596 * [http://www.ebaina.com/bbs/forum.php?mod=viewthread&tid=4858 海思移植opencv+人脸检测] * http://shervinemami.info/faceRecognition.html * [http://ahogrammer.com/2016/11/15/deep-learning-enables-you-to-hide-screen-when-your-boss-is-approaching/ Boss Sensor] * [https://medium.com/@hamedmp/exporting-trained-tensorflow-models-to-c-the-right-way-cf24b609d183 Exporting trained TensorFlow models to C++] * [https://realpython.com/blog/python/face-detection-in-python-using-a-webcam/ Face Detection in Python Using a Webcam] * [http://docs.opencv.org/2.4/modules/contrib/doc/facerec/facerec_tutorial.html Face Recognition with OpenCV] * [http://eyalarubas.com/face-detection-and-recognition.html Face Detection and Recognition (Theory and Practice)] * [https://zhuanlan.zhihu.com/p/24816781 基于深度学习的人脸识别技术综述] * [https://zhuanlan.zhihu.com/p/24837264 谷歌人脸识别系统FaceNet解析] * [https://zhuanlan.zhihu.com/p/25025596 基于mtcnn和facenet的实时人脸检测与识别系统开发] * [https://zhuanlan.zhihu.com/p/24316690 使用深度卷积神经网络方法进行3D人脸重建] * [http://blog.csdn.net/cv_family_z/article/details/51917143 CNN网络二值化--XNOR-Net] <br><br> ==== OpenFace ==== * http://cmusatyalab.github.io/openface/ * https://github.com/cmusatyalab/openface * http://shamangary.logdown.com/posts/800267-openface-installation * [https://github.com/BrandonJoffe/home_surveillance Home surveillance dashboard that uses OpenFace for face recognition] * [https://github.com/pyannote/pyannote-video Face detection, tracking, and clustering in videos using OpenFace] --- [http://nbviewer.jupyter.org/github/pyannote/pyannote-video/blob/master/doc/getting_started.ipynb Getting Start] * [https://github.com/davidsandberg/facenet Face recognition using Tensorflow] * [https://github.com/aybassiouny/OpenFaceCpp C++ implementation for OpenFace library by CMU] <br><br> ==== Facial Keypoints Detection ==== * https://www.kaggle.com/c/facial-keypoints-detection Facial Keypoints Detection - Kaggle project * [https://github.com/dnouri/kfkd-tutorial dnouri's Kaggle Facial Keypoints Detection tutorial] * https://github.com/saber1988/facial-keypoints-detection <br><br> ==== Facial Emotion Recognition ==== * [https://github.com/JostineHo/mememoji A facial expression classification system that recognizes 6 basic emotions: happy, sad, surprise, fear, anger and neutral] * [http://cs231n.stanford.edu/reports2016/022_Report.pdf Facial Emotion Recognition in Real Time] * [https://arxiv.org/pdf/1408.3750.pdf Real-time emotion recognition for gaming using deep convolutional network features] <br><br> ==== Texture Matching ==== * [http://hanzratech.in/2015/05/30/local-binary-patterns.html Texture Matching using Local Binary Patterns (LBP), OpenCV, scikit-learn and Python] <br><br> == Furture == Deep Reinforcement Learning 深度增强学习资源 (持续更新)https://zhuanlan.zhihu.com/p/20885568?refer=intelligentunit 获取人工智能AI前沿信息 https://zhuanlan.zhihu.com/p/21263408?refer=intelligentunit <br><br> == Reference == 问:您对Jeff Hawkins对深度学习的批评有什么看法?Hawkins是On Intelligence一书的作者, 该书2004年出版,内容关于大脑如何工作,以及如何参考大脑来制造智能机器。他声称深度学习没有对时间序列建模。人脑是基于一系列的传感数据进行思考的,人的学习主要在于对序列模式的记忆,比如你看到一个搞怪猫的视频,实际是猫的动作让你发笑,而不是像Google公司所用的静态图片。参见这个链接 答:时间相关的神经网络其实有很多工作,递归神经网络模型对时间关系隐性建模,通常应用于语音识别。比如下面这两个工作。 [1] http://www.cs.toronto.edu/~hinton/absps/RNN13.pdf [2] http://papers.nips.cc/paper/5166 ... neural-networks.pdf 还有这篇文章:http://arxiv.org/abs/1312.6026. 自然语言处理中的序列也有所考虑:http://arxiv.org/abs/1306.2795 问:根据我的理解,深度神经网络训练上的成功跟选取正确的超参数有关系,比如网络深度,隐含层的大小,稀疏约束值等等。有些论文基于随机搜索来寻找这些参数。可能跟代码写得好也有关系。有没有一个地方能让研究者找到某些特定任务的合理超参数呢?在这些参数的基础上,可能更容易找到更优化的参数。 答:可以看上文关于超参数的部分。James Bergstra 继续了这部分工作。我觉得有这么一个数据库,存储着许多推荐的超参数设置,对于神经网络训练是非常有好处的。Github上面的Hyperopt项目,做了类似的事情。hyperopt项目聚焦于神经网络、卷积网络,给出一些超参数设置的建议。以简单的因子分布的形式给出。比如隐含层的数量应该是1到3,每一层的隐含单元数目应该是50到5000。其实超参数还有很多,以及更好的超参数搜索算法等等。下面是更多的参考论文: *http://arxiv.org/abs/1306.2795 *http://arxiv.org/abs/1312.6026 *http://arxiv.org/abs/1308.0850 *http://papers.nips.cc/paper/5166-training-and-analysing-deep-recurrent-neural-networks.pdf 问:Bengio教授,在深度学习中,有那么一类方法,采用比较高级的数学如代数和拓扑集合。John Healy几年前声称通过通过范畴论(Category Theory)改进了神经网络(ART1)。您对于这类尝试有什么看法?是儿戏还是很有前途? 答:可以看看Morton和Montufar的工作,参考附加材料: *http://www.ece.unm.edu/~mjhealy/ *http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.98.6807 热带几何以及概率模型中的热带几何 *http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.242.9890 <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br> <br><br>