bookmark_borderAWS marketplace and no longer supported instance types

I’ve been trying and trying to launch a Neo4j instance on the marketplace without success. It always gave me the nice “success” message, but when I went to the EC2 console: nothing!

A nice green “success” message even though it doesn’t work.

I finally decided to do it the hard way: manually. First step: select an instance type. I immediately try to select the same low cost instance I had picked in the marketplace (m3.medium) and to my surprise that type wasn’t there.

Instance type selection as per the Neo4j marketplace page.

Adding 1 + 1, I went back to the market place and tried with an instance type that still exists: m4.large… success!

So I don’t know who is to blame here, but here are my 2 questions to the internet:

  1. Why do we get the success message even though it doesn’t work?
  2. Why are unsupported instance types offered in the market place?

bookmark_borderPython logging to stackdriver

I recently deployed a python application in google app engine / container engine. When I went to check the logs, everything was logged at the “ERROR” level even though my application uses python logging properly. As far as I know there are 2 ways to fix that:

  1. Use the stackdriver client, which requires an additional dependency and somewhat binds your program to google app engine.
  2. Format the logs in a way that stackdriver can parse them, which is easily configurable.

Continue reading “Python logging to stackdriver”

bookmark_borderRecipe: Testing multiple python versions with pyenv and tox

If you develop a ton of python applications and you need to test under a lot of different versions, and by a lot I mean overlapping major/minor versions (like 3.5.3 and 3.5.4), then a good option is to use pyenv. Along with tox, you can easily test your application against various major/minor versions.

Here is how to do it.
Continue reading “Recipe: Testing multiple python versions with pyenv and tox”

bookmark_borderRecipe: Google App Engine, Cloud SQL and sqlalchemy

Here is a recipe on how I made those thing work together both on a linux development environment and in production. The important thing to remember is: just like recipes for brownies, there are other recipes to achieve the same thing.

The following steps assume you have a Google cloud account with the proper permissions.

Continue reading “Recipe: Google App Engine, Cloud SQL and sqlalchemy”

bookmark_borderFixing Windows 10 sound issue/noise

For a few months I have been struggling with small sound issues. It’s hard to put words to describe it, but it was like noise/static here and there, more often than not while playing games and rendering 3D.

I tried reinstalling audio drivers but that didn’t fix the issue. Finally, I found a post (to which I lost the link) with the solution.

In Windows 10 power options, I changed the configuration from “balanced” to “performance”.
Win10 customize power plan

bookmark_borderSGE/OGS: Clean your logs

At Datacratic, part of our infrastructure runs in the cloud. Our elastic cluster is managed by StarCluster and job dispatch is managed by Open Grid Scheduler (OGS), a fork of Sun Grid Engine (SGE).

While I was looking at StarCluster’s output to help a new user, I realized there was a lot of timeouts. I dug a bit and found out that the command qacct was too slow. Following the lead, I understood that the said command parses a text file each time it is executed.

After a few years of operations, our main cluster has dispatched over 5 000 000 jobs. The log file parsed by qacct was about 2Gb in size. Tada! A couple of search queries taught me that OGS, in its install directory (<path to ogs>/util/logchecker.sh), has a script, ready to be configured, to rotate its logs. I configured and launched it. The timeouts are gone.

Lesson learned: StarCluster/OGS operators, it is important to configure and schedule that script to run every now and then if you want to keep your operations stable.

bookmark_borderC++ / Python Wrapping and system_signal_exception

 

While working on a C++ program that made use of a Python wrapped library to start a subprocess, when the subprocess crashed I got a boost::detail::system_signal_exception at the C++ level even though the executing context was still on the python layer. How is that possible?

Continue reading “C++ / Python Wrapping and system_signal_exception”

bookmark_borderStarCluster: Multiple node instance type support

Last week I released a new feature for the vanilla_improvements branch of StarCluster: multiple instance type support. It means that our cluster can now select the instance type to bid on depending on a configurable factor and the lowest spot market price for each type.

Want to see how it works? Head to the wiki. Want to know more about how I did it? Read further.

Continue reading “StarCluster: Multiple node instance type support”

bookmark_borderPost Ubuntu upgrade: hip-chat not starting (fix)

I have just updated my Ubuntu installation to the latest distribution (15.04) and found out that hip-chat was no longer starting. I went to the terminal to launch it via the command line and had the following error.
%> hipchat
/usr/bin/hipchat: /opt/HipChat/bin/..//lib/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /usr/lib/x86_64-linux-gnu/libicuuc.so.52)

Simpy reinstalling fixed my issue. So first I removed it.
%> sudo apt-get remove hipchat

To reinstall though, I had to re-enable the external dependency commented out by the Ubuntu upgrade procedure.
%> sudo vim /etc/apt/sources.list.d/atlassian-hipchat.list

Remove the starting “#” to re-enable the line. Then run
%> sudo apt-get update
%> sudo apt-get install hipchat

That fixed the problem for me.