Kafka cluster locally - no pain

 Recently developed distributed data science platform using service choreography architectural pattern. Solution assumed Kafka cluster as a message (event) stream provider. I tried different solutions for local Kafka cluster build for docker-compose, but the best one is from Vinsguru

Comments

Post a Comment

Popular posts from this blog

Install Kubeflow locally

For local installation of Kubeflow you can use Vagrant box, Minikube or install it using Multipass. Here I will describe the multipass installation. The best link for the installation process is here . This installation uses Microk8s orchestrator and has some caveats. First of all you have to create volumes manually, other way you services won't work correctly. Before all I would suggest to install version 1.13 of microk8s, because 1.14 is not compatible now with kubflow. sudo snap install microk8s --channel=1.13 For this purpose please edit kubernetes-tools/setup-microk8s.sh line 10: CHANNEL=${CHANNEL:-1.13/stable} 1.14 where changed on 1.13. Then run  sudo kubernetes-tools/setup-microk8s.sh Step 1: Create storage class. I used next data: apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: local provisioner: kubernetes.io/no-provisioner volumeBindingMode: WaitForFirstConsumer Just save and apply this data. kubectl apply -f you_file_name.yaml ...
Keep reading

RabbitMQ and OpenShift

RedHat created very good launcher for a RabbitMQ cluster in OpenShift ( can be found here ). The problem I was stacked a little bit was creation of the cluster in an already created project. I found out that it is easy to do, just change slightly a all.yml file. --- openshift_cluster_content: # No need to create a new project #- object: projectrequest # content: # - name: rabbitmq-spaces # file: "{{ inventory_dir }}/../files/projects/projects.yml" # file_action: create - object: imagestream content: - name: rhel7 file: "{{ inventory_dir }}/../files/imagestreams/images.yml" namespace: scouting # Also important to fix deployment config template, which is located in the files/deployments/template.yml . The deployment will hook wrong image, so cluster won't build without the fix. Problem is in the naming of the image used to build rabbits. image: "${APPLICATION_NAME}:${RABBITMQ_VERSION}" Change to: image: ...
Keep reading