Preface#

Python packages are the fundamental units of shareable code in Python. Packages make it easy to organize, reuse, and maintain your code, as well as share it between projects, with your colleagues, and with the wider Python community. Python Packages is an open-source book that describes modern and efficient workflows for creating Python packages. The focus of this book is overwhelmingly practical; we will demonstrate methods and tools you can use to develop and maintain packages quickly, reproducibly, and with as much automation as possible — so you can focus on writing and sharing code!

Why read this book?#

Despite their importance, packages can be difficult to understand and cumbersome to create for beginners and seasoned developers alike. This book aims to describe the packaging process at an accessible and practical level for data scientists, developers, and programmers. Along the way, we’ll develop a real Python package and will explore all the key elements of Python packaging, including: creating a package file and directory structure, when and why to write tests and documentation, and how to maintain and update your package with the help of automated continuous integration and continuous deployment (CI/CD) pipelines.

By reading this book, you will:

  • Understand what Python packages are, and when and why you should use them.

  • Be able to build your own Python package from scratch.

  • Learn how to document your Python code and packages.

  • Write software tests for your code and automate them.

  • Learn how to release your package on the Python Package Index (PyPI) and discover best practices for updating and versioning your code.

  • Implement CI/CD workflows to build, test, and deploy your package automatically.

  • Get tips on Python coding style, best-practice packaging workflows, and other useful development tools.

Structure of the book#

Chapter 1: Introduction first gives a brief introduction to packages in Python and why you should know how to make them.

Chapter 2: System setup describes how to set up your development environment to develop packages and follow the examples in this book.

In Chapter 3: How to package a Python, we develop an example package from beginning-to-end as a practical demonstration of the key steps involved in the packaging process. This chapter forms the foundation of the book and will act as a reference sheet for readers creating packages in the future.

The remaining chapters then go into more detail about each step in this process, organized roughly in their order in the workflow:

Assumptions#

While this book aims to introduce Python packaging at a beginner level, we assume readers have basic familiarity with the concepts listed in Table 1:

Table 1 Concepts this book assumes readers have basic familiarity with.#

Item

Learning resources

How to import Python packages with the import statement

Python documentation

How to write conditionals (if/elif/else) and loops (for)

Python documentation

How to use and write Python functions

Plotting and Programming in Python: Writing Functions 1

(Optional) Basic familiarity with version control and Git and GitHub (or similar tools)

Happy Git and GitHub for the useR 2 or Research Software Engineering with Python 3

Conventions#

Throughout this book we use foo() to refer to functions, bar for inline commands/variables/function parameters/package names, and __init__.py and src/ to refer to files and directories respectively.

Commands entered at the command line appear as below, with $ indicating the command prompt:

$ mkdir my-first-package
$ cd my-first-package
$ python

Code entered in a Python interpreter looks like this:

import math
round(math.pi, 3)

3.142

Code blocks appear as below:

def is_even(n):
    """Check if n is even."""
    if n % 2 == 0:
        return True
    else:
        return False

If you are reading an electronic version of the book, e.g., https://py-pkgs.org, all code is rendered so that you can easily copy and paste directly from your browser to your Python interpreter or editor.

Persistence#

The Python software ecosystem is constantly evolving. While we aim to make the packaging workflows and concepts discussed in this book tool-agnostic, the tools we do use in the book may have been updated by the time you read it. If the maintainers of these tools are doing the right thing by documenting, versioning, and properly deprecating their code (we’ll explore these concepts ourselves in Chapter 7: Releasing and versioning), then it should be straightforward to adapt any outdated code in the book.

Colophon#

This book was written in JupyterLab and compiled using Jupyter Book. The source is hosted on GitHub and is deployed online at https://py-pkgs.org with Netlify.

Acknowledgments#

We’d like to thank everyone that has contributed to the development of Python Packages. This is an open source book that began as supplementary material for the University of British Columbia’s Master of Data Science program and was subsequently developed openly on GitHub where it has been read, revised, and supported by many students, educators, practitioners and hobbyists. Without you all, this book wouldn’t be nearly as good as it is, and we are deeply grateful. A special thanks to those who have contributed to or provided feedback on the text via GitHub (in alphabetical order of GitHub username): bendichter, benjy765, Carreau, chendaniely, dcslagel, eliasdabbas, fegue, firasm, Kaszanas, Midnighter, mtkerbeR, NickleDave, SamEdwardes, tarensanders, wirthual.

The scope and intent of this book was inspired by the fantastic R Packages 4 book written by Hadley Wickham and Jenny Bryan, a book that has been a significant resource for the R community over the years. We hope that Python Packages will eventually play a similar role in the Python community.