Last week I released a new feature for the vanilla_improvements branch of StarCluster: multiple instance type support. It means that our cluster can now select the instance type to bid on depending on a configurable factor and the lowest spot market price for each type.
Want to see how it works? Head to the wiki. Want to know more about how I did it? Read further.
Continue reading “StarCluster: Multiple node instance type support”
In the core version of StarCluster, when you add many nodes at once (command “addnode -n #”), StarCluster goes through three sequential checks[*] that all nodes need to fulfill in order to move forward and eventually start configuring the nodes within the cluster.
If you add a single node, that’s fine, but if you add 10, you lose some time as the first node might be ready a few minutes before the last node is. That is to say, you are wasting some computing time.
This was a big two weeks for StarCluster as the Datacratic’s version got 3 new features: config on master, multiple subnets support and the dns-suffix option.