Run Horovod with CPU only

I was interested in running federated model training using Horovod in the Kubernetes cluster. The first question I had was - is it possible to run Horovod on CPU, because our containers don't have GPUs.
I found out on GIT repository of Horovod , that this is possible. I have to install within Horovod non-gpu version of Tensorflow, and everything should fly.
Also no need to specify GPU settings in the code, like

config.gpu_options.allow_growth = True
config.gpu_options.visible_device_list = str(hvd.local_rank())

Also, Fardin, advices:

If you have GPUs on your machine and do not want to use them set the environment variable CUDA_VISIBLE_DEVICES=-1.

Comments

Post a Comment

Popular posts from this blog

Install Kubeflow locally

For local installation of Kubeflow you can use Vagrant box, Minikube or install it using Multipass. Here I will describe the multipass installation. The best link for the installation process is here . This installation uses Microk8s orchestrator and has some caveats. First of all you have to create volumes manually, other way you services won't work correctly. Before all I would suggest to install version 1.13 of microk8s, because 1.14 is not compatible now with kubflow. sudo snap install microk8s --channel=1.13 For this purpose please edit kubernetes-tools/setup-microk8s.sh line 10: CHANNEL=${CHANNEL:-1.13/stable} 1.14 where changed on 1.13. Then run  sudo kubernetes-tools/setup-microk8s.sh Step 1: Create storage class. I used next data: apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: local provisioner: kubernetes.io/no-provisioner volumeBindingMode: WaitForFirstConsumer Just save and apply this data. kubectl apply -f you_file_name.yaml ...
Keep reading

RabbitMQ and OpenShift

RedHat created very good launcher for a RabbitMQ cluster in OpenShift ( can be found here ). The problem I was stacked a little bit was creation of the cluster in an already created project. I found out that it is easy to do, just change slightly a all.yml file. --- openshift_cluster_content: # No need to create a new project #- object: projectrequest # content: # - name: rabbitmq-spaces # file: "{{ inventory_dir }}/../files/projects/projects.yml" # file_action: create - object: imagestream content: - name: rhel7 file: "{{ inventory_dir }}/../files/imagestreams/images.yml" namespace: scouting # Also important to fix deployment config template, which is located in the files/deployments/template.yml . The deployment will hook wrong image, so cluster won't build without the fix. Problem is in the naming of the image used to build rabbits. image: "${APPLICATION_NAME}:${RABBITMQ_VERSION}" Change to: image: ...
Keep reading