## Batch Normalization

```import bcolz
def save_array(fname, arr): c=bcolz.carray(arr, rootdir=fname, mode='w'); c.flush()
```

Code snippet from http://course.fast.ai/.

Similar Posts:

## Various ways to create One Hot Encoding

Using numpy:

```In [2]: x = np.array([0, 1, 2, 0, 0])
In [4]: x[:, np.newaxis]
Out[4]:
array([[0],
[1],
[2],
[0],
[0]])

In [5]: np.arange(3) == x[:, np.newaxis]
Out[5]:
array([[ True, False, False],
[False,  True, False],
[False, False,  True],
[ True, False, False],
[ True, False, False]], dtype=bool)

# Just change the boolean to the int.
In [35]: (np.arange(3) == x[:, np.newaxis]).astype(np.float)
Out[35]:
array([[ 1.,  0.,  0.],
[ 0.,  1.,  0.],
[ 0.,  0.,  1.],
[ 1.,  0.,  0.],
[ 1.,  0.,  0.]])
```

Using sklearn:

```In [8]: x
Out[8]: array([0, 1, 2, 0, 0])

In [9]: from sklearn.preprocessing import OneHotEncoder

# Reshape changes shape of the x to (5, 1).
In [10]: x.reshape(-1, 1)
Out[10]:
array([[0],
[1],
[2],
[0],
[0]])

# Return value of fit_transform() is csr_matrix. todense() changes it to numpy matrix.
In [11]: OneHotEncoder().fit_transform(x.reshape(-1, 1)).todense()
Out[11]:
matrix([[ 1.,  0.,  0.],
[ 0.,  1.,  0.],
[ 0.,  0.,  1.],
[ 1.,  0.,  0.],
[ 1.,  0.,  0.]])
```

Using keras’s numpy util.

```In [12]: x
Out[12]: array([0, 1, 2, 0, 0])

In [13]: from keras.utils.np_utils import to_categorical
In [14]: to_categorical(x, len(set(x)))
Out[14]:
array([[ 1.,  0.,  0.],
[ 0.,  1.,  0.],
[ 0.,  0.,  1.],
[ 1.,  0.,  0.],
[ 1.,  0.,  0.]])
```

Similar Posts:

## Linear Regression using Keras

This is example code to perform linear regression using keras.

```import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import SGD

import numpy as np
from numpy.random import random

x = random((30, 2))
w = np.array([3., 2.])
b = 1.
y = np.dot(x, w) + b
print x
print y
model = Sequential()
# 30 observations. Each observation has 2 features.
# MSE because we want linear regression.
model.compile(optimizer=SGD(lr=0.1), loss='mse')
model.fit(x, y, nb_epoch=60, batch_size=1)
print model.get_weights()
```

Output:

```Using Theano backend.
Using gpu device 0: GeForce GTX 1080 (CNMeM is disabled, cuDNN 5105)
[[ 0.66716875  0.44536398]
[ 0.58253148  0.45428442]
[ 0.49094032  0.77897022]
[ 0.16345683  0.27807741]
[ 0.29913647  0.87666064]
[ 0.41150342  0.4329423 ]
[ 0.30724994  0.56194767]
[ 0.1448368   0.61572276]
[ 0.89288341  0.45568394]
[ 0.20008006  0.1446671 ]
[ 0.58501705  0.1407299 ]
[ 0.8924096   0.58803216]
[ 0.76954948  0.95146172]
[ 0.17315788  0.03576668]
[ 0.01515587  0.36599027]
[ 0.77232613  0.35686848]
[ 0.4897022   0.52092717]
[ 0.50756237  0.20100097]
[ 0.8372522   0.53871228]
[ 0.2223611   0.5919245 ]
[ 0.89898591  0.24163213]
[ 0.571022    0.2140571 ]
[ 0.55041835  0.00383233]
[ 0.08253098  0.64526628]
[ 0.3512973   0.53963146]
[ 0.73578765  0.65469051]
[ 0.91344962  0.40350727]
[ 0.74023006  0.34414037]
[ 0.41329666  0.22543498]
[ 0.82787326  0.41838276]]
[ 3.8922342   3.65616327  4.03076141  2.0465253   3.6507307   3.10039487
3.04564516  2.66595593  4.59001811  1.88957438  3.03651096  4.85329311
5.21157187  1.591007    1.77744815  4.03071534  3.51096094  2.92468906
4.58918115  2.8509323   4.18022197  3.14118018  2.65891972  2.53812548
3.13315483  4.51674398  4.54736341  3.90897091  2.69075993  4.32038531]
Epoch 1/60
30/30 [==============================] - 0s - loss: 1.6538
Epoch 2/60
30/30 [==============================] - 0s - loss: 0.1377
Epoch 3/60
30/30 [==============================] - 0s - loss: 0.0793
Epoch 4/60
30/30 [==============================] - 0s - loss: 0.0370
Epoch 5/60
30/30 [==============================] - 0s - loss: 0.0265
Epoch 6/60
30/30 [==============================] - 0s - loss: 0.0146
Epoch 7/60
30/30 [==============================] - 0s - loss: 0.0096
Epoch 8/60
30/30 [==============================] - 0s - loss: 0.0043
Epoch 9/60
30/30 [==============================] - 0s - loss: 0.0030
Epoch 10/60
30/30 [==============================] - 0s - loss: 0.0020
Epoch 11/60
30/30 [==============================] - 0s - loss: 0.0012
Epoch 12/60
30/30 [==============================] - 0s - loss: 6.9193e-04
Epoch 13/60
30/30 [==============================] - 0s - loss: 3.6242e-04
Epoch 14/60
30/30 [==============================] - 0s - loss: 2.2359e-04
Epoch 15/60
30/30 [==============================] - 0s - loss: 1.1410e-04
Epoch 16/60
30/30 [==============================] - 0s - loss: 7.5656e-05
Epoch 17/60
30/30 [==============================] - 0s - loss: 4.6557e-05
Epoch 18/60
30/30 [==============================] - 0s - loss: 2.9460e-05
Epoch 19/60
30/30 [==============================] - 0s - loss: 1.6638e-05
Epoch 20/60
30/30 [==============================] - 0s - loss: 1.0647e-05
Epoch 21/60
30/30 [==============================] - 0s - loss: 6.4342e-06
Epoch 22/60
30/30 [==============================] - 0s - loss: 3.5493e-06
Epoch 23/60
30/30 [==============================] - 0s - loss: 1.8375e-06
Epoch 24/60
30/30 [==============================] - 0s - loss: 1.3024e-06
Epoch 25/60
30/30 [==============================] - 0s - loss: 8.3916e-07
Epoch 26/60
30/30 [==============================] - 0s - loss: 5.3163e-07
Epoch 27/60
30/30 [==============================] - 0s - loss: 2.8679e-07
Epoch 28/60
30/30 [==============================] - 0s - loss: 1.5040e-07
Epoch 29/60
30/30 [==============================] - 0s - loss: 1.1201e-07
Epoch 30/60
30/30 [==============================] - 0s - loss: 6.0981e-08
Epoch 31/60
30/30 [==============================] - 0s - loss: 4.7074e-08
Epoch 32/60
30/30 [==============================] - 0s - loss: 2.9919e-08
Epoch 33/60
30/30 [==============================] - 0s - loss: 1.6059e-08
Epoch 34/60
30/30 [==============================] - 0s - loss: 9.3970e-09
Epoch 35/60
30/30 [==============================] - 0s - loss: 5.7633e-09
Epoch 36/60
30/30 [==============================] - 0s - loss: 3.3312e-09
Epoch 37/60
30/30 [==============================] - 0s - loss: 2.1822e-09
Epoch 38/60
30/30 [==============================] - 0s - loss: 1.2432e-09
Epoch 39/60
30/30 [==============================] - 0s - loss: 6.8956e-10
Epoch 40/60
30/30 [==============================] - 0s - loss: 4.4050e-10
Epoch 41/60
30/30 [==============================] - 0s - loss: 2.5711e-10
Epoch 42/60
30/30 [==============================] - 0s - loss: 1.5499e-10
Epoch 43/60
30/30 [==============================] - 0s - loss: 8.9069e-11
Epoch 44/60
30/30 [==============================] - 0s - loss: 4.6494e-11
Epoch 45/60
30/30 [==============================] - 0s - loss: 3.0863e-11
Epoch 46/60
30/30 [==============================] - 0s - loss: 1.5283e-11
Epoch 47/60
30/30 [==============================] - 0s - loss: 8.9050e-12
Epoch 48/60
30/30 [==============================] - 0s - loss: 4.8648e-12
Epoch 49/60
30/30 [==============================] - 0s - loss: 3.5887e-12
Epoch 50/60
30/30 [==============================] - 0s - loss: 2.0066e-12
Epoch 51/60
30/30 [==============================] - 0s - loss: 1.1189e-12
Epoch 52/60
30/30 [==============================] - 0s - loss: 6.5086e-13
Epoch 53/60
30/30 [==============================] - 0s - loss: 3.3727e-13
Epoch 54/60
30/30 [==============================] - 0s - loss: 1.7621e-13
Epoch 55/60
30/30 [==============================] - 0s - loss: 9.8529e-14
Epoch 56/60
30/30 [==============================] - 0s - loss: 5.6370e-14
Epoch 57/60
30/30 [==============================] - 0s - loss: 5.3054e-14
Epoch 58/60
30/30 [==============================] - 0s - loss: 4.9738e-14
Epoch 59/60
30/30 [==============================] - 0s - loss: 3.9790e-14
Epoch 60/60
30/30 [==============================] - 0s - loss: 4.4527e-14
[array([[ 3.        ],
[ 1.99999952]], dtype=float32), array([ 1.00000024], dtype=float32)]
```

Similar Posts:

## Bay Area Deep Learning School 2016

장장 20시간의 강의가 유튜브에 올라와 있습니다.

그리고 http://www.bayareadlschool.org/schedule 에 보시면 발표 자료가 있습니다.

Similar Posts:

## reshape2 vs tidyr

같은 듯 다른 듯한 두 패키지를 잘 비교한글이 있어 링크합니다.

Comparison of the reshape2 and tidyr packages

tidyr은 어떤일이 벌어지는지 하나하나 적는 형식이라 단계별로 알기 쉽고 함축이 없어 명확한 반면, reshape은 좀 더 선언적이라 코드가 짧고 하고자 하는 일 자체를 더 잘 설명하지 않나 싶습니다. 코드가 선언적이어야하는지 절차적이어야하는지 이 두가지는 항상 업치락뒤치락 하면서 같이 가는 것 같습니다.

Similar Posts:

## word2vec in tensorflow

공부하면서 참고한 자료들 올려봅니다.

Tensorflow tutorial
word2vec_basic.py에 대한 주석붙인 설명
CBOW와 Skip-gram의 차이
Xin Rong, word2vec Parameter Learning Explained. Context word가 여러개일때 어떻게 훈련하는가. 결론은 입력도 출력도 평균을 사용한다는 것.

Similar Posts:

## Playing go

인공지능 바둑에 대한 글들을 모아보려합니다.

The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions

Similar Posts:

## LeNet-5

LeNet-5, convolutional neural networks

LeNet-5 is our latest convolutional network designed for handwritten and machine-printed character recognition.

Similar Posts:

## Optimization algorithms

다양한 알고리즘의 동작 모습

Visualizing Optimization Algos

Similar Posts: