Building and pushing to Packagr from Gitlab CI

Packagr can be easily integrated with GitLab CI. So, if you use GitLab as your git repository, you can easily use GitLab CI to build and deploy your packages to Packagr whenever you push your changes.

The folder structure

For the purposes of this tutorial, we'll be assuming that your Python code's folder structure looks something like this:

my_repository/
    my_package/
        __init__.py
    setup.py

And that your setup.py file looks something like this:

from setuptools import setup

setup(name='my_package',
      version='0.1.0',
      description='Some Python package',
      author='Me',
      author_email='me@example.com',
      license='MIT',
      packages=['my_package'],
      zip_safe=False)

Updating your .gitlab-ci.yml file

Gitlab CI depends on there being a file called .gitlab-ci.yml at the root location of your repository in order to work. This file defines stages that will be executed whenever code is pushed to your repository, if certain conditions are met.

If you don't already have one, add a file called .gitlab-ci.yml to the root of your project. Let's define a stage called package by adding the following content to your file:

stages:
  - package

package:
  stage: package
  image: python:3.6-alpine
  script:
    - pip install twine wheel 
    - python setup.py sdist bdist_wheel
    - twine upload dist/*

The above script will run the package stage whenever you push a commit to your repository. This stage installs the twine and wheel packages using pip, creates wheel/tarball format builds, then uploads those packages to Packagr.

However, the above will not work as is, as we have not told twine to upload to our specific Packagr repository, or provided any credentials. To do this we need to go to Gitlab > Settings > CI/CD > Environmental variables and set the three following environmental variables as follows:

  • TWINE_REPOSITORY_URL: Your Packagr reposiotry URL
  • TWINE_USERNAME: Your Packagr email address
  • TWINE_PASSWORD: Your Packagr password

GitLab CI will now attempt to build and deploy your code to Packagr every time you push your commit

About versioning

The example we have created so far assumes that you are manually managing your versioning, and setting your version number correctly before pushing your commits. If you don't update your version number before every push with this approach, then twine will return a 409 error due to a version conflict.

To avoid this, we recommend using git tag to manage your versioning. To do this, first update your .gitlab-ci.yml file to only run the package stage on tagged commits, as follows:

stages:
  - package

package:
  stage: package
  image: python:3.6-alpine
  script:
    - pip install twine wheel 
    - python setup.py sdist bdist_wheel
    - twine upload dist/*
  on:
    - tags

And edit your setup.py file as follows:

from setuptools import setup
import os

setup(name='my_package',
      version=os.environ['CI_COMMIT_TAG'],
      ...
      )

This will use Gitlab CI's built in CI_COMMIT_TAG environmental variable to set your version based on the git tag. You can now deploy your code as follows:

git commit -m "Some changes"
git tag 0.2.0
git push --tags

## Using Poetry

If you use Poetry to build your packages, then you can use poetry instead of twine to publish your code. In this case, your .gitlab-ci.yml file would look something like this:

package:
  stage: package
  image: python:3.6-alpine
  script:
    - curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python
    - source $HOME/.poetry/env 
    - poetry config repositories.packagr $PACKAGR_REPOSITORY_URL
    - poetry config http-basic.packagr $PACKAGR_EMAIL $PACKAGR_PASSWORD
    - poetry publish -r packagr --build

In this case, you'd need to set your $PACKAGR_REPOSITORY_URL, $PACKAGR_EMAIL and $PACKAGR_PASSWORD environmental variables in GitLab accordingly