Dec 26

I had the need to refactor a python package, better said, I had to rename the package. The package in question is using namespaces, which complicates the matter, as this implies that there are two steps to the process: you have to exchange all references to the module name and you have to re-arrange the package structure. Assuming you’re at the toplevel of your (otherwise clean) package directory, the following shows the manual steps involved in the process.

First we rearrange the directories:

# rearrange directories
mv old new
cd new
mv secondlevel newseclevel
# cd pkgname
# mv pkgname newname

Of course, this might not be a good idea if you’re using a version control system. I’m using Mercurial, so instead of the steps above, I’m using the following steps shown below. If you’re using something else, you need to substitute the appropriate incantation. Please note, I’m not committing here, but would do so in a final step, after everything worked out as expected — at this stage, after the shuffling around, the code wouldn’t work, so it doesn’t make sense to commit.

# rename directories with Mercurial
hg rename old new
hg rename new/secondlevel new/newseclevel

After re-arranging the directory structure, we now need to substitute any imports in the code which still use the (no longer functional) package structure. If you’re using any install script (e.g. setup.py in case of using EasyInstall), don’t forget to modify that accordingly.

cd new

# Caveat: if you use partial imports (e.g. import old.secondlevel.*) this would not be caught
# refactor all mentions of the exact package name, in Python code and in text files (e.g. doctest)
find . -type f -name '*.py' -exec perl -i.bak -pane 's/old.secondlevel.pkgname./new.newseclevel.pkgname/g' {} \;
find . -type f -name '*.txt' -exec perl -i.bak -pane 's/old.secondlevel.pkdname/new.newseclevel.pkgname/g' {} \;


# back to top
cd ../
# or cd ../../ if you had to rename a third level as well

# run pylint / coverage / etc.
nosetests --with-coverage --with-xunit --cover-package=new.newseclevel --cover-erase
pylint -f parseable new.newseclevel | tee pylint.out

# delete all .bak files
find old -type f -name '*.bak' -exec rm { }\;

# modify setup.py or some such 
# $EDITOR setup.py

# if using Mercurial, commit now
# hg commit -m 'Renamed package from old.secondlevel.pkgname to new.newseclevel.pkgname'.

This is, of course, ugly on two behalves: it’s a brittle manual process and even worse, it’s using Perl to do the actual substitution. But if I resort to shell, Perl’s concise syntax is hard to beat. So, maybe I’ll write a small Python program to overcome this (even the integration with Mercurial would be straight-forward, given that Mercurial is written in Python). However, the manual process has one benefit: it’s very easy to validate each step and ensure that nothing went wrong.

Oh, and of course you need to update all the code that still refers to your old package structure …

Posted by Holger Schauer

Defined tags for this entry: ,

0 Trackbacks

  1. No Trackbacks

1 Comments

Display comments as(Linear | Threaded)
  1. Holger Schauer says:

    I asked in the Python community on google plus on what other people use. Some people seem to use build-in functionality from Eclipse or WingIDE, but I also got pointed to Rope, the refactoring library for python: http://rope.sourceforge.net/

Add Comment


Markdown format allowed
Enclosing asterisks marks text as bold (*word*), underscore are made via _word_.
E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.
CAPTCHA