Posts

Showing posts from 2018

Flask parse multiple file upload

Assume you have a form with: <input type="file" .... multiple> This strategy can be often use in Angular >2.  Angular Material does not support input type file, so to enable file upload we catch click mouse event on usual material input control and trigger click on hidden input type file. If you would like to upload multiple files with this strategy into Flask server, it will be hold in ImmutableMultiDict ( more ).   This dictionary can store multiple values for the same key. For example: ImmutableMultiDict([('files', <FileStorage: '[Edward_Thorp]_Beat_the_marke.pdf' ('application/pdf')>), ('files', <FileStorage: '[Edward_Thorp]_The_mathematics_of_gambling.djvu' ('image/vnd.djvu')>), ('files', <FileStorage: ['Larry_Dressler]_Roger_Schwarz_Standing_in_the_F.pdf' ('application/pdf')>)]) If you would try ...

OpenShift Airflow Cluster

I spent some time playing with Airflow installation in OpenShift. It was a little bit tricky. Python is sensitive to run own application under not root user, as requires OpenShift Dedicated. Every pod in OpenShift Dedicated is running under not root user.   Finally, I am happy with the result. You can find it following this link . Also, made changes into the Puckel Docker image for Airflow. Created own Docker image on DockerHub. You can find the image here .

Casting ServletRequest to HttpServletRequest

Sometimes is a need to convert ServletRequest object into HttpServletRequest object, because, for example, we need more metadata about a request to get from. There is a simple way to do that (taken from Josh Long's course): // Let's check if request is of type HttpServletRequest Assert.isTrue(servletRequest instanceof HttpServletRequest, "this assumes you have an HTTP request"); // Now do casting HttpServletRequest httpServletRequest = HttpServletRequest.class.cast(servletRequest);

RabbitMQ and OpenShift

RedHat created very good launcher for a RabbitMQ cluster in OpenShift ( can be found here ). The problem I was stacked a little bit was creation of the cluster in an already created project. I found out that it is easy to do, just change slightly a all.yml file. --- openshift_cluster_content: # No need to create a new project #- object: projectrequest # content: # - name: rabbitmq-spaces # file: "{{ inventory_dir }}/../files/projects/projects.yml" # file_action: create - object: imagestream content: - name: rhel7 file: "{{ inventory_dir }}/../files/imagestreams/images.yml" namespace: scouting # Also important to fix deployment config template, which is located in the files/deployments/template.yml . The deployment will hook wrong image, so cluster won't build without the fix. Problem is in the naming of the image used to build rabbits. image: "${APPLICATION_NAME}:${RABBITMQ_VERSION}" Change to: image: ...

OANDA

Today started my acquaintance with online trading platform  OANDA . They give nice possibility to build and test trading bots. There is special place for developers. To start development I use Build Trading Bots in Java book. Unfortunately, API described in the book is deprecated now, however that's not a problem. I am looking for logic more.

Persisting a detached entity in JPA

Read topic here

Jenkins pipeline clone and publish Git commit

I have been in a situation when a need to trigger CI/CD build on let's say repository B , on successful run of CI/CD build of repository A . It is worth to mention that CI/CD pipeline is built on OpenShift layer and executed by Jenkins server. So, what we have repository a with data science models, and repository B with microservice logic to execute these models (prediction, recommendation). To avoid data scientist to merge code from one repo to another I decided to use a Jenkins pipeline. Let's start from the beginning: the first step is to clone you second repository ( B) , if you will do it with command like: // Clone repo git url: 'https://github.com/jglick/simple-maven-project-with-tests.git' ... It won't work. Yes it will clone a repository, however it will overwrite you cloned repository A even, if you try to do it from different folder. That's why for cloning I used checkout functionality: // Clone repo checkout([$class: 'GitSCM',...

Golang Structs Caveats

By default Go initiates empty structs (as well as other variable types) with empty fields. The question is next - let's assume we have next struct: type Example struct { ready bool, num int16, q float32, } Total 1 byte for bool, 2 bytes for int16 and 2 for float32 = 7 bytes.? Not so, because we have int16 after bool, Go will use padding - use memory in same amount as next declared field requires. That means if we have field that requires 2 bytes after field that requires 1 byte, Go will require 1 more byte as a padding for "1 byte" field. If we use fields like int64 - 8 bytes, on every object we will spend extra 7 bytes. What is the solution? Declare "heavy" fields first, than "light" ones. type Example struct { num int16, q float32, ready bool, }

How to create read only user in PostgreSQL

-- Create a group CREATE ROLE readaccess; -- Grant access to existing tables GRANT USAGE ON SCHEMA public TO readaccess; GRANT SELECT ON ALL TABLES IN SCHEMA public TO readaccess; -- Grant access to future tables ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO readaccess; -- Create a final user with password CREATE USER tomek WITH PASSWORD 'secret'; GRANT readaccess TO tomek; Taken from gist.github.com

Implementation marketplace using Stripe or PayPal

Definition Marketplace – type of business model of Internet application, which provides service for sellers to sell products directly in application, and application will capture and charge buyer using payment provider systems. It is possible for a marketplace application to take platform fees. Overview There are different platforms which allow to implement marketplace functionality. Most commonly used are PayPal Marketplace and Stripe Connect , also recently become popular Adyen . Only PayPal and Stripe was investigated. All of the have developer’s sandboxes. Paypal has separate one, Stripe included into a real account (You have to enable test data, just toggle “View test data”). Settings Stripe To be able to work with Stripe application should have Publishable key and Secret key and Client id / Test mode client id. Publishable key and secret key you can found under Developers -> API keys . Client Id is received from an account manager. ...

Biochemistry - That's Cool

Image
Recently visited biochemistry site of BI in Vienna. Production of proteins using cells is really interesting area for me. Don't plan to switch areas, however process is complex, and am really keen to get knowledge how people manage it.

Kafka auto flush

To avoid a call of flush command on Kafka producer just set property linger.ms to 1 on properties passed to a Kafka producer constructor.