AWS marketplace and no longer supported instance types

I’ve been trying and trying to launch a Neo4j instance on the marketplace without success. It always gave me the nice “success” message, but when I went to the EC2 console: nothing!

A nice green “success” message even though it doesn’t work.

I finally decided to do it the hard way: manually. First step: select an instance type. I immediately try to select the same low cost instance I had picked in the marketplace (m3.medium) and to my surprise that type wasn’t there.

Instance type selection as per the Neo4j marketplace page.

Adding 1 + 1, I went back to the market place and tried with an instance type that still exists: m4.large… success!

So I don’t know who is to blame here, but here are my 2 questions to the internet:

  1. Why do we get the success message even though it doesn’t work?
  2. Why are unsupported instance types offered in the market place?
Aside

Python logging to stackdriver

I recently deployed a python application in google app engine / container engine. When I went to check the logs, everything was logged at the “ERROR” level even though my application uses python logging properly. As far as I know there are 2 ways to fix that:

  1. Use the stackdriver client, which requires an additional dependency and somewhat binds your program to google app engine.
  2. Format the logs in a way that stackdriver can parse them, which is easily configurable.

Continue reading

Lecture: Machine Learning in Action

Après un peu plus de 3 années passées chez Datacratic, il était grand temps que je m’intéresse davantage à l’apprentissage machine. C’est le livre Machine Learning in Action que j’ai choisi pour me lancer.

Continue reading

Machine Learning in Action Book Cover Machine Learning in Action
Peter Harrington
Computers
Manning Publications
2011-12
354

Provides information on the concepts of machine theory, covering such topics as statistical data processing, data visualization, and forecasting.

Fixing Windows 10 sound issue/noise

For a few months I have been struggling with small sound issues. It’s hard to put words to describe it, but it was like noise/static here and there, more often than not while playing games and rendering 3D.

I tried reinstalling audio drivers but that didn’t fix the issue. Finally, I found a post (to which I lost the link) with the solution.

In Windows 10 power options, I changed the configuration from “balanced” to “performance”.
Win10 customize power plan

SGE/OGS: Clean your logs

At Datacratic, part of our infrastructure runs in the cloud. Our elastic cluster is managed by StarCluster and job dispatch is managed by Open Grid Scheduler (OGS), a fork of Sun Grid Engine (SGE).

While I was looking at StarCluster’s output to help a new user, I realized there was a lot of timeouts. I dug a bit and found out that the command qacct was too slow. Following the lead, I understood that the said command parses a text file each time it is executed.

After a few years of operations, our main cluster has dispatched over 5 000 000 jobs. The log file parsed by qacct was about 2Gb in size. Tada! A couple of search queries taught me that OGS, in its install directory (<path to ogs>/util/logchecker.sh), has a script, ready to be configured, to rotate its logs. I configured and launched it. The timeouts are gone.

Lesson learned: StarCluster/OGS operators, it is important to configure and schedule that script to run every now and then if you want to keep your operations stable.