Python

"Alternatives" (caveat emptor)

At the time of writing the system version of Python is often 2.7, whereas newer applications benefit from using Python 3.*. One way to deal with that is to include "env" in hashbang pointing to the exact version you want to use. Apache/WSGI deployments may require additional footwork to ensure the correct version of Python runtime is used in mod_wsgi etc.

Debian "Alternatives" - Debian has a way to specify the default version of an app. For example, if more than one version of Python is present on the system, the command "update-alternatives" can be used to activate any of the available choices.

Caution - it's not a good idea to switch from the version of Python which came with your distro, since there documented and undocumented dependencies in various places, on that particular version. Random things may break such as software update, applications like Dropbox etc. Caveat Emptor.

Remove an alternative version:

sudo update-alternatives --remove python /usr/bin/python3

Example above allows to fall back on the previous version, such as Python 2.7.

It is recommended that instead of replacing the default, relevant scripts contain explicit reference to version 3+ if possible.

Building Python from source

Certain applications (e.g. mod_wsgi) require Python to use shared libraries. Python (like 3.5) needs to be rebuilt for that:

./configure --enable-shared
make altinstall

If you expect that your applications will have a dependency on sqlite, another option must be added:

--enable-loadable-sqlite-extensions

...which should be done after

sudo yum install sqlite-devel

Building from source may also be required for other reasons. If you got your system completely bare, you will need to install gcc before you can compile Python.

Please see notes below to understand what you need to do to ensure that the Python you are building has requisite support for ssl, zlib etc.

In user space:

./configure --prefix=$HOME/python
make && make install

Also see:

https://janikarhunen.fi/how-to-install-python-3-5-1-on-centos-7.html

pip3

The pip utility most often needs to be run under "sudo". There are some issues with that as explained below.

Certain versions of sudo (on some Linux distributions) "reset the environment" in order to assure security. Most variables are unset. This may make installation work cumbersome. Policies that govern that are contained in the file /etc/sudoers. CAUTION - it should really only be edited with the "visudo" utility which checks for syntax. If that file becomes invalid you may lose all of sudo functionality which in some cases is the only way to have access to root privileges. This will effectively "brick" the system. Then, there are exceptions to rule of preserving certain variables even if you do edit the "sudoers" file. The variable LD_LIBRARY_PATH is notoriously clobbered no matter what you try. The way around it is to supply the value on the command line, and more than one can be included. Example:

sudo LD_LIBRARY_PATH=/usr/local/lib /usr/local/bin/python3.5 get-pip.py

There are caveats when running the command listed above. Zlib is needed, so

 
sudo yum install zlib-devel

Also, it needs ssl support in Python. So after the following steps (on CentOS)

sudo yum install openssl
sudo yum install openssl-devel

...one has to rebuild Python as described in the previous section.

Filesystem

Absolute path

import os
os.path.abspath("../myfile.txt")

shutil

The "shutil" package contains a plethora of useful methods to copy and modify files. Watch out for file metadata, this may be subtle. For example, there are differences between "copy" and "copyfile". Copying directories may require a complete definition of the target path.

JSON

By default deserializing JSON data produces dictionaries with arbitrary order of keys. To enforce the same ordering in the data structure as exists in the source JSON, one should use a directive as illustrated below:

data = json.loads('{"foo":1, "bar": 2}', object_pairs_hook=OrderedDict)