Manipulating OS functionalities using Python

os module: directory and file manipulation

The os module is a portable way of using operating system dependent functionality. With os, we can basically execute the Linux commands Basic Unix/Linux Commands from Python. For a more complete list please see https://docs.python.org/3.7/library/os.path.html.

Current directory

>>> import os
>>> os.getcwd()
'/Users/ylee/Documents/ucsc/18_fall/ams129/playground/python'

List a directory

>>> os.curdir
'.'
>>> os.listdir(os.curdir)
['ch05_python_exception_debugging.rst', 'ch05_python_oop.rst', 'ch05_python_IO.rst', 'ch05_python_libraries.rst']

Change to another directory

>>> path = '../'
>>> os.chdir(path)
>>> os.getcwd()
'/Users/ylee/Documents/ucsc/18_fall/ams129/playground'

Make a directory

>>> os.mkdir('codes')
>>> 'codes' in os.listdir(os.curdir)
True

Rename the directory

>>> os.rename('codes','example_codes')
>>> 'codes' in os.listdir(os.curdir)
False
>>> 'example_codes' in os.listdir(os.curdir)
True

>>> os.rmdir('example_codes')
>>> 'example_codes' in os.listdir(os.curdir)
False

Delete a file

>>> fp = open('junk.txt','w')
>>> fp.close()
>>> 'junk.txt' in os.listdir(os.curdir)
True
>>> os.remove('junk.txt')
>>> 'junk.txt' in os.listdir(os.curdir)
False

os.path: path manipulations

os.path provides common operations on pathnames

>>> fp = open('junk.txt','w')
>>> fp.close()

>>> a = os.path.abspath('junk.txt') # displays an absolute path from the top most directory, as opposed to showing a relative one
>>> a
'/Users/ylee/Documents/ucsc/18_fall/ams129/playground/junk.txt'

>>> os.path.split(a)
('/Users/ylee/Documents/ucsc/18_fall/ams129/playground', 'junk.txt')

>>> os.path.dirname(a)
'/Users/ylee/Documents/ucsc/18_fall/ams129/playground'

>>> os.path.basename(a)
'junk.txt'

>>> os.path.splitext(os.path.basename(a))
('junk', '.txt')

>>> os.path.exists('junk.txt')
True

>>> os.path.isfile('junk.txt')
True

>>> os.path.isdir('junk.txt')
False

Running Linux commands

You can also run Linux commands Basic Unix/Linux Commands using os.system

>>> cmd1 = 'ls -l'
>>> os.system(cmd1)
total 0
drwxr-xr-x  5 ylee  staff  160 May  3 19:34 fortran
-rw-r--r--  1 ylee  staff    0 May 29 18:33 junk.txt
drwxr-xr-x  4 ylee  staff  128 May 29 17:43 python
drwxr-xr-x  5 ylee  staff  160 May 14 12:25 sphinx
0

With os.system you can indeed perform almost everything within a directory

>>> cmd2 = 'touch foo'
>>> os.system(cmd2)
0
>>> os.system(cmd1)
total 0
-rw-r--r--  1 ylee  staff    0 May 29 18:37 foo
drwxr-xr-x  5 ylee  staff  160 May  3 19:34 fortran
-rw-r--r--  1 ylee  staff    0 May 29 18:33 junt.txt
drwxr-xr-x  4 ylee  staff  128 May 29 17:43 python
drwxr-xr-x  5 ylee  staff  160 May 14 12:25 sphinx
0

glob: pattern matching on files

The glob module provides convenient file pattern matching, which is same as ls *.keyword. For example, if we wish to find all files that have .rst file extensions, in Linux you would do

$ ls *rst
ch05_python_IO.rst                  ch05_python_exception_debugging.rst ch05_python_oop.rst       ch05_python_libraries.rst

From Python, you can do the same as

>>> import glob
>>> dir(glob)
['__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_glob0', '_glob1', '_glob2', '_iglob', '_ishidden', '_isrecursive', '_iterdir', '_rlistdir', 'escape', 'fnmatch', 'glob', 'glob0', 'glob1', 'has_magic', 'iglob', 'magic_check', 'magic_check_bytes', 'os', 're']
>>> glob.glob('*.rst')
['ch05_python_exception_debugging.rst', 'ch05_python_oop.rst', 'ch05_python_IO.rst', 'ch05_python_libraries.rst']

sys module: system-specific information

The sys module provides system-specific information related to the Python interpreter.

To see which version of Python you are running and where it is installed

>>> import sys
>>> sys.platform
'darwin'

>>> sys.version
'3.6.5 (default, Apr 25 2018, 14:23:58) \n[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)]'

>>> sys.prefix
'/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6'

We already saw that sys.path is a list of strings that specifies the search path for modules. Initialized from PYTHONPATH

>>> sys.path
 ['', '/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6', '/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload', '/Users/ylee/Library/Python/3.6/lib/python/site-packages', '/usr/local/lib/python3.6/site-packages', '/Users/ylee/src/jupyter-CAF-kernel/prebuild/jupyter-caf-kernel'Users]

Exercise

  1. Write a simple routine that removes those junk_1.txt, …, junk_n.txt files you created in the exercise problems from the previous section Recap & Outlook.