Passion is like genius; a miracle.

  • LLM의 한계에 대하여

    LLM이 인간의 모든 지식을 담을 수 없다는 흥미로운 글을 읽었다. 인공지능이 인간의 지식을 다 표현할 수 있는가 아닌가는 상당히 오래된 주제이지만, 이 글의 열정적인 톤과 풍부한 사례는 무척 재미있었다. 예를들어 글에서는 performer (예를들어 피겨 스케이팅 선수) 의 perform 하는 기술은 글로 기술되지 않으며 글로 기술된다 하였다고 해도 그것이 그대로 그 퍼포먼스를 따라할 수 있단 이야기는…

    Tags:

    ,

  • Positional encoding for timeseries data

    This is positional encoding for timseries data inspired by A Transformer-based Framework for Multivariate Time Series Representation Learning. Assuming that the input is (batch_size, sequence_length, feature_dim) and each feature is a float, the way we add positional encoding is: This is applied like: See PositionalEncoding of keras_nlp here for generic implementation.

    Tags:

    ,

  • A trick to ignore a data in categorical cross entropy of keras

    When there’s a data point that you want to ignore in loss computation, you can use ignore_class parameter in tf.keras.losses.SparseCategoricalCrossentropy. But the same parameter doesn’t exist in tf.keras.losses.CategoricalCrossentropy. I don’t know why, but that’s troublesome when there’s needs. Even the SparseCategoricalCrossEntropy’s ignore_class isn’t easy to use either since it requires one to add a class…

    Tags:

  • Structural matching simplifies the code

    Well known structural matching in python would be match case. But there’s an other kind and it checks # of items, e.g., If foo() returns [1], it works fine. But if it returns either multiple items [1, 2, 3] or an empty list, it fails. This is pretty neat. I have been checking the length…

    Tags:

  • Better dict for configuration

    When I use dict as a config of deep learning algorithms, I often run multiple training runs, varying the config value. As a trivial example: And then I run multiple iterations varying the ‘lr’. When doing that, I feel worried about making mistake like updating the dict with wrong key, e.g., To address such concerns,…

    Tags:

    ,

  • Tuple is immutable in python

    I find this very interesting. These examples are from fluent python. Likewise… The reason of these behaviors are easier to understand if we take a look at the methods of tuple. Whenever += is called for tuple, say, a += b, what’s actually happening is a + b and then a = a + b.…

    Tags:

  • Keras 에서 Jax, Flax 로 코드를 옮겨본 소감

    성능이 더 개선된다는 이유로, 그리고 기술적인 호기심으로 Keras 코드를 Jax로 옮기고 있다. 그러면서 느낀 점들을 공유해서 같은 migration을 하고 싶은 경우에 참고가 되었으면 한다. Jax 의 첫인상은 배우기 어렵단 것이었다. 특히 Sharpbits 에 그 내용이 잘 정리되어있는데 대부분은 데이터가 immutable 하다는 것으로 요약된다. 이것이 처음에는 매우 당혹스럽다가 서서히 익숙해졌다. 그러나 몇가지는 생각지도 못한 문제가 있었다.…

    Tags:

  • Forcing accessibility font size and dark theme on mobile chrome

    Chrome on Android has accessibility settings to override font size. But there are websites that the change doesn’t apply. Also, chrome doesn’t use dark theme even when system theme is using it. Enable these two flags at chrome://flags to fix them. Restarting Chrome, you’ll see new accessibility settings for fonts and new menu item on…

    Tags:

  • Use left option key as ALT for VIM on mac

    Use iterm2 and change the left option to ESC+ Once that done, left option will work as alt in the vim. But it will break your cursor movement in the shell, i.e., opt+left and opt+right won’t move cursor over words. To fix that, type in ‘cat’, type the key you want to bind, i.e., opt+left…

    Tags:

  • Use nerd font on chromebook terminal and changing the font size

    Press ctrl+shift+j in the terminal, then paste the below. See https://gist.github.com/izzqz/af36c5bbde67ffff0f7614cf738d75a5 for details. Plus, you can change font size. This is from https://chromium.googlesource.com/apps/libapps/+/HEAD/nassh/docs/FAQ.md#How-do-I-use-web-fonts p.s. It still doesn’t render all the icons, e.g,. vim-tree in nvim. I don’t know answer for that.

    Tags: