Posts

Showing posts from 2019

Solving service discovery: Node.js microservices without a registry - Ri...

Image

Building Distributed Systems with Node.js and ØMQ

Image

Choreographing Microservices - Allen Holub

My impressions about the presentation. I watched it with strong motivation to find a way for my project, a recommendation process for a chain of process unit steps. A recommendation is computed on a combination of ML models, which are dynamically created during the time. I guess service choreography is good way to go for me. I like conception of no contract, no implementation aware solutions. Everything is transmitted in the system through events. However, I am not sure is it possible to implement such an architecture with Python. Will keep searching. Also, interesting point is in the building communication schema using stickers of different colors.

Berlin Buzzwords 2019: Szilard Pafka–Better than Deep Learning: Gradient...

Image

Berlin Buzzwords 2019: Jon Bratseth–Scalable machine-learned model serving

Image
https://docs.vespa.ai/documentation/vespa-quick-start.html https://github.com/vespa-engine/vespa/tree/master/storageapi/src/vespa/storageapi/message

Tensorflow Serve | multiple models same server

There is a way to serve multiple models within one Tensorflow Serve server providing config file. model_config_list: { config: { name: "model1", base_path: "/models/model1", model_platform: "tensorflow" }, config: { name: "model2", base_path: "/models/model2", model_platform: "tensorflow" } } More detailed information possible to get here . What I would like to know how convenient is this solution? Is it possible to upload new versions? Edit config?

Serving Models in Production with TensorFlow Serving (TensorFlow Dev Sum...

Image

TensorFlow Extended An End to End Machine Learning Platform for TensorFlow

Image

Really good articles on built_value

Image
https://medium.com/dartlang/darts-built-value-for-immutable-object-models-83e2497922d4#.48dyezxcl https://medium.com/dartlang/darts-built-collection-for-immutable-collections-db662f705eff https://medium.com/dartlang/darts-built-value-for-serialization-f5db9d0f4159#.h12y94wu7

Speaker Diarization

Found super interesting article about speaker diarization. Guys build full framework to separate speakers on audio online. The paper can be found here . The repository is here . Other interesting articles on topic: https://www.infoq.com/news/2018/11/Google-AI-Voice/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5996252/ http://theconversation.com/human-voices-are-unique-but-our-study-shows-were-not-that-good-at-recognising-them-79520 https://towardsdatascience.com/automatic-speaker-recognition-using-transfer-learning-6fab63e34e74 https://medium.com/@ageitgey/machine-learning-is-fun-part-6-how-to-do-speech-recognition-with-deep-learning-28293c162f7a https://medium.com/linagoralabs/voice-activity-detection-for-voice-user-interface-2d4bb5600ee3 https://medium.com/linagoralabs/computing-mfccs-voice-recognition-features-on-arm-systems-dae45f016eb6 https://towardsdatascience.com/beginners-guide-to-speech-analysis-4690ca7a7c05 https://towardsdatascience.com/speech-recognition-i...

Flutter Course

Image
From my point of view very nice course for free. I highly appreciate the work of Nick. Clean code practises in course are really good.

Really the best Belgian waffles

We found this receipt today with my middle daughter. Waffles were so delicious :)

2014 Personality Lecture 11: Existentialism: Viktor Frankl

Image

Blob Url Video

Interesting ideas found here  and here . Cool solution  youtube-dl .

Flutter application to learn

Want to get to know Flutter well - here is GIthub repository with the application to learn.

Voice to Text

Spinx Open-source application for voice-to-text processing. It is written on C++ and Java, Python client exists. Code can be found here . Project page .

Implementing Microservices Security Patterns

Image

Good articles about LSTM and Attention

LSTM Attention

Vulnerability audit

Here is a list of sites, helping estimate vulnerabilities  of own site: https://report-uri.com https://securityheaders.com/

Backpropagation the best explanations

https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ https://www.youtube.com/watch?v=d14TUNcbn1k&t=2273s

Speech and AI

Here below provided a bunch of articles about speech processing and noise suppression:   Audio features for web-based ML  - nice article about in-browser speech recognition.   Simple Audio Recognition  - article about Tensorflow usage for audio classification.   Real-Time Noise Suppression Using Deep Learning  - article about noise suppression   Denoise Speech Using Deep Learning Networks  - Matlab version  RNNoise: Learning Noise Suppression  - Mozilla project (useful) GitHub denoising projects: Wavenet For Speech Denoising Source-Denoising-Pix2Pix-cGAN

Abeer Alwan — Voice Feature Extraction from Smartphones

Image

Install Kubeflow locally

For local installation of Kubeflow you can use Vagrant box, Minikube or install it using Multipass. Here I will describe the multipass installation. The best link for the installation process is here . This installation uses Microk8s orchestrator and has some caveats. First of all you have to create volumes manually, other way you services won't work correctly. Before all I would suggest to install version 1.13 of microk8s, because 1.14 is not compatible now with kubflow. sudo snap install microk8s --channel=1.13 For this purpose please edit kubernetes-tools/setup-microk8s.sh line 10: CHANNEL=${CHANNEL:-1.13/stable} 1.14 where changed on 1.13. Then run  sudo kubernetes-tools/setup-microk8s.sh Step 1: Create storage class. I used next data: apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: local provisioner: kubernetes.io/no-provisioner volumeBindingMode: WaitForFirstConsumer Just save and apply this data. kubectl apply -f you_file_name.yaml ...

Convolutional networks

Best article: https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/ Best free video: https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv https://www.youtube.com/watch?v=ArPaAX_PhIs&list=PLkDaE6sCZn6Gl29AoE31iwdVwSG-KnDzF Channel https://www.youtube.com/channel/UCcIXc5mJsHVYTZR1maL5l9w

Parallelizing Scientific Python with Dask | SciPy 2018 Tutorial | James ...

Image

Kubernetes (Openshift) secret in a Jenkins slave

Image
Recently I created data - science model training and upload CI/CD pipeline in OpenShift. I had next idea: Data-scientist creates new model and uploads it in a repository. A repository triggers web-hook and notifies OpenShift about pushed / merged commit. Jenkins starts an execution of commands described in Jenkinsfile. Jenkinsfile in my case contains list of commands to create a container, train a model there, test it, wrap it and then synchronize it  with a responsible Jenkins slave (slave started the process). Then slave uploads an artifact in to a system.  The only problem I met implementing this solution was injecting certificates into a Jenkins slave. I tried different ideas - mounting secrets as volumes to a shared folder in Jenkins master, using Kubernetes plugin form for attaching secrets... - I didn't get any result. Help, I received from my colleague Michael, solution is quite simple, but not easy to found. Create a special type of 'ssh-auth' ...

x.509 certificate with Spring

https://www.youtube.com/watch?v=_HfWDtu7A5I and https://stackoverflow.com/questions/48111084/authentication-by-certificate-for-webflux and  https://blog.codecentric.de/en/2018/08/x-509-client-certificates-with-spring-security/

Build m2o relations in SqlAlchemy with a composite FK

Solution is well defined in the next article.

Spring boot trailing slashes match

Spring boot uses special configuration to set up trailing slashes, which is an overridden method of WebMvcConfigurer - configurePathMatch. More information can be found here .

How to run Python tests with Jenkins pipeline

From my point of view good solution is to use virtualenv there. Below is small snippet of code for a Jenkinsfile. sh """ pip install --user virtualenv && virtualenv venv && . \$(pwd)/venv/bin/activate && pip install -r src/test_requirements.txt && export PYTHONPATH="src" && cd src/tests && nosetests -v """

Hibernate lazy load issue

When you use entities with lazy loading fields you can meet Jaxson error, like " No serializer found....". The best solution and description I have found is here.

Architecture Deep Dive in Spring Security - Joe Grandja @ Spring I/O 2017

Image