Posts

Showing posts from December, 2021

RuntimeError: Building llvmlite requires LLVM 11.x.x, got '7.0.1'.

 I researched recently transformers library, especially Wa2Vec2 model. The main goal was to build a prototype of automatic speech recognition (ASR) on RaspberryPI 4 (RSP). There were a few caveats how to make the whole ecosystem operational. However here I would like to describe how to deal with quite common error related to Numba installation. Numba and llvmlite packages are dependencies for sklearn package, which is quite often used in machine learning world.  In this topic I will describe how to install Numba and llvmlite as well as how to deal with the next error: RuntimeError: Building llvmlite requires LLVM 11.x.x, got '7.0.1'. Be sure to set LLVM_CONFIG to the right executable path. First of all, let me explain what is the problem. While installing Numba Python compiles it's code and for this purpose it needs LLVM compiler. In the error message the build process notifies us that it has found not satisfying version of LLVM compiler. To check your version you can...

Optax - optimization library for JAX

JAX based libraries become more and more popular. I think guys from Google (Deepmind) picked right direction to optimize numpy functionality and build a top the whole ML infrastructure. As example of the good picked direction we have Optax  Python library which has different optimizers for JAX based solutions. The repository has good introduction examples (e.g. this one ), however good knowledge of JAX is required.  More on JAX

pcm2float

 I would like to share different implementations of Pulse-code modulation (PCM) to float conversion. Usually we receive PCM in int16 type, but some models expect normalized float64. The logic for conversion is not comlex and even so I have managed to find so interesting options. The basic one logic looks like this: def pcm2float(sig, dtype=np.float64):s sig = np.asarray(sig) # make sure it's a NumPy array assert sig.dtype.kind == 'i', "'sig' must be an array of signed integers!" dtype = np.dtype(dtype) # allow string input (e.g. 'f') # Note that 'min' has a greater (by 1) absolute value than 'max'! # Therefore, we use 'min' here to avoid clipping. return sig.astype(dtype) / dtype.type(-np.iinfo(sig.dtype).min) The code is taken from StackOverflow answer on a question  SciPy wavfile: music in, garbage out ? What it does? I think the most important is in the line. Let's split it on the log...

Wavelets

 Sometime ago I participated in BI data science competition. The main task was to find the best algorithm for audios with velcro crackle classification. The winner was algorithm which used wavelet transform. This link leads to a good article about wavelets using Python ( GitHub repo ).

Just a picture

Image
 Just a picture, drawn by me, illustrating a complex process ML models deployment for the company biochemists.

Install Onnxruntime on RaspberryPI 4

 Currently I have found two ways to get ONNX runtime on RaspberryPI . The first one is a Docker image, and the second one is a compilation on own RaspberryPI. I prefer the second one ( link ). Everything is nicely described but I think it is important to know more tools. 1. screens - Some of the compilation commands can run multiple hours, such that your ssh connection can be interrupted, and you will lose a process. To avoid this, you can use screen ( good tutorial link ) 2. I use Jupyter to test my solutions, as well as virtual environments. To make virtual environment accessible as Jupyter kernel, please check this article . 3. Maybe you will need system packages in you virtual environment. In this case next advice from StackOverflow helps: Create the environment with  virtualenv --system-site-packages  . Then, activate the virtualenv and when you want things installed in the virtualenv rather than the system python, use  pip install --ignore-installed  or...

UX maturity and values

Image
  Interesting topic about UX maturity was raised by Natalie Hangson in her blog post  UX Maturity Models – A Collection . She listed all known maturity schemas and characterized them from company management point of view. The whole topic, from my take is slightly political, and usually takes a lot of time to pick right model for a company. however from educational point I would recommend this topic. The second, UX at scale values TED presentation maid by Margaret Gould is super motivating. I would suggest not only UX designers but all specializations working on a product.

D3.js alternatives

 Recently looked for a good alternatives for D3.js. I had some experience with it and find the library awesome. D3.js gives a lot of power, but as usual with power comes responsibility to develop charts from very basic components. It gives a lot of flexibility, however usually I often use general charts. So I decided to create a priority list of charts javascript libraries.   1. plotly.js   - just love it. I use Plotly Python package as well. Interface is understandable. 2. ApexCharts.js  - this library I would select second. It is simple to use, has nice color scheme by default. 3. Nivo.js  - super cool library., has a lot of components. Supports SVG and Canvas. 3. Chart.js  - simple library easy to use. 4. Google Charts  - charts library from Google. Plenty of options. Google's style :) 3. JS 3. 

XCode on Linux

Image
 Just navigating the Internet found to interesting solutions: The first on is Sosumi . It comes as Snap package.  Project on GitHub The second project, is Docker-OSX . Project on GitHub