The Microsoft Cognitive Toolkit (CNTK)

正苦於 Linux 不太熟， GPU 還用不上的情境之下，
無意中發現 MicroSoft 也加入了 openSource 的戰局，提出以下的 ToolKit 想與 Theano 及 TensorFlow 抗衡。試玩了一下，好像還不錯，至少幫我成功用上了 GPU。

而且還順便玩到了 CIFAR-10 影像辨認資料庫，以及 sequence-to-sequence translation。
這2項正是一直想拿來當作 進階版的 PR 教程，放在這裡也順便當作本學期 PR 課程的 happy ending。



The Microsoft Cognitive Toolkit  (CNTK)

presentation @ ICS2016

Machine Translation of EnglishIdentifiers in Python Programs into TraditionalChinese

Back-Propagation Algorithm in Deep Learning Neural Network

2016 期中考

https://www.dropbox.com/s/2ymsxa4tb9og8hu/ryBackProp2016.pdf?dl=0

Convolutional Neural Networks

http://cs231n.github.io/convolutional-networks/

CS231n Convolutional Neural Networks for Visual Recognition

ch12_multilayer_perceptron (mlp) , Backpropagation

ch12.zip


ch12.dir

PR2016_01

今年開始活用 GitHub





rasbt 的教科書及 程式碼

1. TextBook:  https://www.packtpub.com/mapt/book/all_books/9781783555130,  ch_all_2p

2. code:  https://github.com/rasbt/python-machine-learning-book

3. slides: https://github.com/dmitriydligach/PyMLSlides

投影片pdf :

https://goo.gl/9Vd2q3


日文翻譯，及專欄導讀:

ブック・インサイド―『Python機械学習プログラミング』学び方ガイド

以上的材料非常珍貴, 但要如何活用呢。

1. 準備工具: Anaconda, Notepad++,  Acrobat Reader 

2. 把教科書開在一旁 也建議用紙本印出。可記筆記。

3. 把 jupyter 開在一旁, 雖時跑一跑程式碼的結果, 改一改實驗一下。

呂老師的 講義
https://github.com/renyuanL/ryPR2016


ryPR2016.zip

------------------------------------------

本課程要求，

1. 每周來上課(第2周起每周點名)。

2. 爭取機會上台報告(第3周起)

3. 期中考試(第10周)、可以上台口頭外加書面報告取代。

4. 期末(第15周開始)專題報告。

PR2015

Django 初體驗，
即上手。

先試玩一下！

日文之注音、分詞及解析。

以及 音文同步 (未完成)

http://163.25.101.243

原始碼在此，

https://www.dropbox.com/s/epq3y89tzbw5k3r/rySite004.zip?dl=0

準備動作：

1. 安裝： django
        > pip install django

2. 安裝： MeCab
        (坦白說，這個不大好裝，
        我的電腦是 Windows 10, 有 VC++
        我在 Python 3.4.3, 成功，
        我試過 Python 3.5.x, 失敗！)
        https://www.dropbox.com/s/zduv2nxs6m6vibj/成功於Python343之mecab-python-0.996-windows-build-master.zip?dl=0


3. 安裝：romkan
        ( 也得小小的麻煩，但比起 MeCab 算簡單的了)

4. 以及其他的、、、

老師在這裡無法百分百詳述過程，
有興趣同學可帶電腦來問我。


Django 之簡單講解，就以老師做的題目為例：

https://www.dropbox.com/s/4bep62qen6zxsh7/ryReadme001.py?dl=0

https://www.dropbox.com/s/m7s3rkwgh971o0w/ryReadme002.py?dl=0

其他更詳細請參考：


https://djangogirlstaipei.gitbooks.io/django-girls-taipei-tutorial/content/


這個參考也不錯

Python Tutorial 第四堂（1）Django 起步走

http://www.codedata.com.tw/python/python-tutorial-the-4th-class-1-django-getting-started/

基本上跟隨上述2個照做，再加上一點點除錯、嘗試，
我用大約4小時，把文字型，非 GUI 圖形介面，建立起來，
資料已可傳來傳去，算是有初步的網路伺服器功能。

還可前往 官網，英文要好一點就是了！ 

https://www.djangoproject.com/start/

ryPR002: scikit-learn, Machine Learning in Python

We will use
Python + scikitLearn.



http://scikit-learn.org/stable/index.html

scikit-learn

Machine Learning in Python

• Simple and efficient tools for data mining and data analysis
• Accessible to everybody, and reusable in various contexts
• Built on NumPy, SciPy, and matplotlib
• Open source, commercially usable - BSD license

ryPR001

名為 "Pattern Recognition" (樣型識別) 的這門課，是一個更大學門稱做 "Machine Learning" 的前行課程，入門請先欣賞吾友 Sebastian Raschka 的文章：

How Machine Learning Algorithms Work Part 1

http://sebastianraschka.com/Articles/2015_singlelayer_neurons.html

In the above article, the author says:

Machine learning is one of the hottest and most exciting fields in the modern age of technology. 

Thanks to machine learning, we enjoy 

• robust email spam filters, 
• convenient text and voice recognition, 
• reliable web search engines, 
• challenging chess players, and, hopefully soon, 
• safe and efficient self-driving cars.

ryPatternRecognition001

開始準備 Pattern Recognition 課程教材

第一彈：

http://rasbt.github.io/mlxtend/

http://rasbt.github.io/mlxtend/docs/classifier/neuralnet_mlp/

https://pbs.twimg.com/media/CIT6J0nWEAAf56u.png

TermScores in 2014 Autumn class

TermScore= Mid*2.5+(oral + doc)*2.5 +29

			Mid		oral	doc		TermScore
			期中考		期末口頭報告	期末書面		學期成績
資工系碩士班	M0229030	DHARAMJEET	17		4	4		92
資工系碩士班	M0329005	辛沛勳	18		2	3		87
資工系碩士班	M0329007	邱思綺	18		3	3		89
資工系碩士班	M0329009	牟光邦	15		3	3		82
資工系碩士班	M0329010	陳陽	15		3	3		82
資工系	B0029013	謝松樺	19		2	2		87
資工系	B0029016	林書聿	18		2	3		87
資工系	B0029028	鄭逸軒	20		4	4		99
資工系	B0029043	毛為翰	19.5		4	3		95
資工系	B0029054	邱筠婷	18.5		3	3		90
資工系	B9929021	簡立芃	14.5		2	2		75
資工系	B9929036	蔡崇元	15		2	2		77

PR 2014, Final Project schedual

PR 2014, Final Project schedual

(15) (2015/01/02) --> (2014/12/27)

No class in the classroom, 

please watch the audio processing tutorial video on the youtube.
And think about your final term projects
I will be in the office during 9:00~12:00,
If you have any questions, please come to meet me.
or leave a message in this page.

(16) Final Term Report: MNist Task (2015/01/09) ，

Undergraduate (大學部) students can go to attend the 專題展，

Graduate (研究所) students should come to class to do the final report

Each student has 30 minutes to talk and demo.

You should prepare a power point file (.ppt) to do the presentation.

a word document (.doc) as a formal document

and the source code of Python to demonstrate your research.

all are put in your blogspot,

and leave the message in THIS page to tell me.

The number behind your name is your scores in the mid-term exam.

(the full score is 20)

The report schedule is as follows, please come to class at 9:10 on time.

資工系碩士班	M0229030	DHARAMJEET	17
資工系碩士班	M0329005	辛沛勳	18
資工系碩士班	M0329007	邱思綺	18
資工系碩士班	M0329009	牟光邦	15
資工系碩士班	M0329010	陳陽	15

(17) Final Term Report: MNist Task (2015/01/16)

The number behind your name is your scores in the mid-term exam.

(the full score is 20)

The report schedule is as follows, please come to class at 9:10 on time.

資工系	B0029013	謝松樺	19
資工系	B0029016	林書聿	18
資工系	B0029028	鄭逸軒	20
資工系	B0029043	毛為翰	19.5
資工系	B0029054	邱筠婷	18.5
資工系	B9929021	簡立芃	14.5
資工系	B9929036	蔡崇元	15

(18) Final Term Report: MNist Task (2015/01/23)

This is a buffer time slot

and also the deadline for all documents to be updated.

Getting Audio-Visual Data for pattern-recognition

Getting Audio-Visual Data for pattern-recognition
Audio by Microphone
Video by Camera

https://github.com/renyuanL/realTimeSpectrogram

In addition to
pygame
http://www.lfd.uci.edu/~gohlke/pythonlibs/#pygame

scipy-stack
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy-stack

you also need:

pyaudio

http://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio







a very good collection about Audio Processing Tutorial
edited by my friend Yuan-Fu Liao

Please watch it and get some idea 
about your final project on the PR course

Digit Recognition by mouse gestures

File to be printed out for discussing today
https://www.dropbox.com/s/7ux2zdde6o1msuj/mouseGesture.pdf?dl=0


I have not had time to prepare for SVM theory presentation yet.
So I found a demo to share with you, which I shared with students last year.

It is an interesting pygame program,
which do the recognition of mouse movements (gesture)

The source code can be found here:
https://pypi.python.org/pypi/MooseGesture/0.9.0


From it you can learn how to capture the mouse movement at first.

To run it, don't forget to install pygame first, if you havn't done it, please go to 

• pygame:  pygame‑1.9.2a0.win32‑py3.4.exe

to get and install pygame.



The pattern recognition algorithm I adopted last year was very very simple,
even simpler than the Perceptron.


This is my modification to a digit recognition.

ryMoosegesture 滑鼠動作辨識
http://goo.gl/2R8LzE

It is very primitive. You students can be inspired by this modification and modify it yourselves to become your final project.

It is time you start for your final project....

By the way, I have just finished your mid-term scoring (期中考成績)

If you want to view it, please come to class.

---

By the way, I have a lovely file to share with you. Hope you like it.
My suggestion: if you crack out some of these 18 turtle programs,
you will make great progress in Python programming.

---

Perhaps this is useful to understand the original turle.py

Precision and recall

Precision and recall

http://en.wikipedia.org/wiki/Precision_and_recall

Definition (classification context)

For (binary) classification tasks, the terms

true positives, 

true negatives, 

false positives, 

and false negatives 

(see alsoType I and type II errors) 

compare the results of the classifier under test with trusted external judgments. 

Gaussian mixture models (GMM)

2.1. Gaussian mixture models

http://scikit-learn.org/stable/modules/mixture.html#gaussian-mixture-models

Support Vector Machine (SVM)

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/readings/MIT6_034F10_svm.pdf

Lecture 16: Learning: Support Vector Machines

http://en.wikipedia.org/wiki/Support_vector_machine

ryMnistClassifiers

In this blog, I have succeeded in doing a classification task on MNist database.

Based on sklearn, I have tried Perceptron, svm, and gmm.
In addition to those, I also found a mlp.py in Github and debug it to run in Python 3.

So I have 4 types of classifiers now.

At first, I hope to make things as simple as possible, so I tend to use the simplest setting for each classifiers, usually using their default values.

Since the MNist database is not small, so I haven't used all of it at first,
Because there are so many parameters to be set, doing some smaller test is necessary,

So I only take 1/10 of MNist dataset to train all the classifiers.

you can see that in the program ryMnistClassifiers03.py ,

Use Scikit-learn to do Pattern Recognition

Use Scikit-learn to do Pattern Recognition on the MNist Database of handwritten digits


To install Scikit-learn on Windows , You should intall numpy, scipy and matplotlib, first, it seems a little bit tedious. One convenient way to install those 3 commonly used packages, I suggest, is just  go to this site. http://www.lfd.uci.edu/~gohlke/pythonlibs/

Find 2 exe files

• Scipy-stack: Scipy‑stack‑14.8.27.win32‑py3.4.exe
• Scikit-learn: scikit‑learn‑0.15.2.win32‑py3.4.exe

After installation, just check it in Python

>>> import sklearn

Then goto the homepage of Scikit-learn, to get some idea of it.

Here are some pre-jobs I have done after 2 days study. Just for your reference. Enjoy it.

By the way, if you want to construct a real-time system, you need use the camera to capture a real image. You can use pygame module, which can be found here.

• pygame:  pygame‑1.9.2a0.win32‑py3.4.exe

PR 2014, HomeWork schedual

From today to finalterm, please do some homeworks and then final report.
You should post your homework on some webpage and tell me the link on this page

when post your message here, please include your student ID number and name.
you should make sure the link you provide is good.
my suggestion is use www.blogspot.com as the teacher does.

(Week) Task

(9) Mid term (Today, 2014/11/21)
(10) Python HW01
(11) (2014/12/05)

(12) Python HW02 (12/12)
        to read the MNist database and show it on the screen.

(13) (12/19)

(14) Python HW03 (12/26)
        to use the bpnn.py program to do the MNist database.

(15) (2015/01/02)
(16) (2015/01/09)
(17) Final Term Report: MNist Task (2015/01/16)
(18) Final Term Report: MNist Task (2015/01/23)

Python HW01 is about running and modifying 3 programs in Turtle graphics.





Python HW02 is about how to read the MNist database and show it on the screen.

Python HW03 is about how to use the bpnn.py program to do the MNist database.

The Final term project is an overall integration of Pattern Recognition on MNist database.

You can search anything or use any packages (numpy, scikit, ...) to do this task.

old task material about mnist database

old task material about mnist database

https://dl.dropboxusercontent.com/u/33089565/ryTeach2013.html