Information Technology
If you develop a ton of python applications and you need to test under a lot of different versions, and by a lot I mean overlapping major/minor versions (like 3.5.3 and 3.5.4), then a good option is to use pyenv. Along with tox, you can easily test your application against various major/minor versions.
Here is how to do it.
Continue reading “Recipe: Testing multiple python versions with pyenv and tox”
Here is a recipe on how I made those thing work together both on a linux development environment and in production. The important thing to remember is: just like recipes for brownies, there are other recipes to achieve the same thing.
The following steps assume you have a Google cloud account with the proper permissions.
Continue reading “Recipe: Google App Engine, Cloud SQL and sqlalchemy”
For a few months I have been struggling with small sound issues. It’s hard to put words to describe it, but it was like noise/static here and there, more often than not while playing games and rendering 3D.
I tried reinstalling audio drivers but that didn’t fix the issue. Finally, I found a post (to which I lost the link) with the solution.
In Windows 10 power options, I changed the configuration from “balanced” to “performance”.
At Datacratic, part of our infrastructure runs in the cloud. Our elastic cluster is managed by StarCluster and job dispatch is managed by Open Grid Scheduler (OGS), a fork of Sun Grid Engine (SGE).
While I was looking at StarCluster’s output to help a new user, I realized there was a lot of timeouts. I dug a bit and found out that the command qacct was too slow. Following the lead, I understood that the said command parses a text file each time it is executed.
After a few years of operations, our main cluster has dispatched over 5 000 000 jobs. The log file parsed by qacct was about 2Gb in size. Tada! A couple of search queries taught me that OGS, in its install directory (<path to ogs>/util/logchecker.sh), has a script, ready to be configured, to rotate its logs. I configured and launched it. The timeouts are gone.
Lesson learned: StarCluster/OGS operators, it is important to configure and schedule that script to run every now and then if you want to keep your operations stable.
While working on a C++ program that made use of a Python wrapped library to start a subprocess, when the subprocess crashed I got a boost::detail::system_signal_exception at the C++ level even though the executing context was still on the python layer. How is that possible?
Continue reading “C++ / Python Wrapping and system_signal_exception”
I was surprised this morning when I realized that the call to sys.exit is not really exiting. It is instead raising exception SystemExit which, if left unhandled, will exit with the proper exit code.
Continue reading “Python sys.exit: a suggestion”
Last week I released a new feature for the vanilla_improvements branch of StarCluster: multiple instance type support. It means that our cluster can now select the instance type to bid on depending on a configurable factor and the lowest spot market price for each type.
Want to see how it works? Head to the wiki. Want to know more about how I did it? Read further.
Continue reading “StarCluster: Multiple node instance type support”
I have just updated my Ubuntu installation to the latest distribution (15.04) and found out that hip-chat was no longer starting. I went to the terminal to launch it via the command line and had the following error.
%> hipchat
/usr/bin/hipchat: /opt/HipChat/bin/..//lib/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /usr/lib/x86_64-linux-gnu/libicuuc.so.52)
Simpy reinstalling fixed my issue. So first I removed it.
%> sudo apt-get remove hipchat
To reinstall though, I had to re-enable the external dependency commented out by the Ubuntu upgrade procedure.
%> sudo vim /etc/apt/sources.list.d/atlassian-hipchat.list
Remove the starting “#” to re-enable the line. Then run
%> sudo apt-get update
%> sudo apt-get install hipchat
That fixed the problem for me.
There should be a rule for all parsers: Parsers “print” method should always render a string that can be parsed back without changing the semantics. In pseudo code, it translates to:
initial_string = "parse me" //parse back assert to_string(parse(initial_string)) == initial_string //don't change the semantics assert parse(to_string(parse(initial_string) == parse(initial_string)
If I do the same with JavaScript and JSON:
var jsonStr = '{"key1":"val1","key2":"val2"}';
JSON.stringify(JSON.parse(jsonStr)) == jsonStr;
As programmers, the less we need to think and worry, the better. Parsers following that rule can be used with confidence; they will never betray you. If that statement doesn’t convince you of the importance for consistency, let me give you a couple of examples of errors caused by inconsistent parsers/printers.
Continue reading “Parsers printing rule: make sure you print what you parsed”
Trivia: What is the difference between the encoded query string parameter “a+b” and “a%20b” ?
Answer: Nothing! They are both encoded representations for “a b”.
Isn’t a “+” supposed to remain a “+”? Well, the URL and the query strings are not encoded following the same rules. In the URL, the “+” remains a “+” indeed, but in the query string it’s actually encoded and becomes a “%2B”. This can be misleading.
Continue reading “Query string white space vs plus”