Continuous Delivery for automating package releases (npm, cookbooks, gems, pip, jars, etc)
CapsuleCD
WARNING: CAPSULECD IS DEPRECATED It's been replaced by Packagr, a series of small composable tools, with all the same functionality: https://github.com/PackagrIO/docs
CapsuleCD is a generic Continuous Delivery pipeline for versioned artifacts and libraries written in any language. Its goal is to bring automation to the packaging and deployment stage of your library release cycle. CapsuleCD is incredibly flexible, and works best when implemented side-by-side with a CI pipeline.
A short list of the features...
*.lock
files/ shrinkwrap files with new version.gitignore
)Introduction What is CapsuleCD
CapsuleCD is a generic Continuous Delivery pipeline for versioned artifacts and libraries written in any language. Its goal is to bring automation to the packaging and deployment stage of your library release cycle. It automates away all the common steps required when creating a new version of your library.
Why use CapsuleCD
At first glance, it seems simple to publish a new library version. Just bump the version number and publish, right? Well, not always:
CapsuleCD handles all of that (and more!) for you. It pretty much guarantees that your library will have proper and consistent releases every time. CapsuleCD is well structured and fully tested, unlike the release scripts you've manually cobbled together for each library and language. It can be customized as needed without rewriting from scratch. The best part is that CapsuleCD uses CapsuleCD to automate its releases. We dogfood it so we're the first ones to find any issues with a new release.
How do I start?
You can use CapsuleCD to automate creating a new release from a pull request or from the latest code on your default branch.
Automated pull request processing:
Here's how to use docker to merge a pull request to your Ruby library
docker run \
-e CAPSULE_SCM_GITHUB_ACCESS_TOKEN=123456789ABCDEF \
-e CAPSULE_SCM_REPO_FULL_NAME=AnalogJ/gem_analogj_test \
-e CAPSULE_SCM_PULL_REQUEST=4 \
-e CAPSULE_RUBYGEMS_API_KEY=ASDF12345F \
AnalogJ/capsulecd:ruby capsulecd start --scm github --package_type ruby
Or you could download the latest linux release, and call CapsuleCD directly to merge a pull request to your Python library:
CAPSULE_SCM_GITHUB_ACCESS_TOKEN=123456789ABCDEF \
CAPSULE_SCM_REPO_FULL_NAME=AnalogJ/pip_analogj_test \
CAPSULE_SCM_PULL_REQUEST=2 \
CAPSULE_PYPI_USERNAME=AnalogJ \
CAPSULE_PYPI_PASSWORD=mysupersecurepassword \
capsulecd start --scm github --package_type python
Creating a branch release
TODO: add documentation on how to create a release from the master branch without a pull request. Specify the env variables required.
Engine
Every package type is mapped to an engine class which inherits from a EngineScm
class, ie EnginePython
, EngineNode
,
EngineRuby
etc. Every SCM type is mapped to a SCM class, ie ScmGithub
. When CapsuleCD starts, it initializes the
specified Engine, and loads the correct SCM module. Then it begins processing your source code step by step.
Step Description
pipeline_init_step This will initialize the SCM client, ensuring that we can authenticate with the git server
scm_retrieve_payload_step If a Pull Request # is specified, the payload is retrieved from SCM api, otherwise the repo default branch HEAD info is retrived.
scm_process_pull_request_payload or scm_process_push_payload Depending on the retrieve_payload step, the merged pull request is cloned, or the default branch is cloned locally
assemble_step Code is built, which includes adding any missing files/default structure, version bumping, etc.
dependencies_step Download package dependencies
compile_step Optional compilation of source into binaries
test_step Run the package test runner(s) (eg. npm test, rake test, kitchen test, tox), linter, formatter & dependency vulnerbility scanner
package_step Clean any unnecessary files, commit any local changes and create a git tag. Nothing should be pushed to remote repository
dist_step Push the release to the package repository (ie. npm, chef supermarket, rubygems)
scm_publish Push the merged, tested and version updated code up to the source code repository. Also do any source specific releases (github release, asset uploading, etc)
Configuration
Specifying your GITHUB_ACCESS_TOKEN
and PYPI_PASSWORD
via an environmental variable might make sense, but do you
really want to specify the PYPI_USERNAME
, REPO_FULL_NAME
each time? Probably not.
CapsuleCD has you covered. We support a global YAML configuration file (that can be specified using the --config_file
flag), and a repo specific YAML configuration file stored as capsule.yml
inside the repo itself.
Setting Inheritance/Overrides
CapsuleCD settings are determined by loading configuration in the following order (where the last value specified is used)
--config-file
)capsule.yml
)CAPSULE_
)Configuration Settings
Check the example.capsule.yml
file for a full list of all the available coniguration options.
As mentioned above, all settings can be specified via Environmental variable. All you need to do is convert the setting
to uppercase and then prefix it with CAPSULE_
. So pypi_password
can be set with CAPSULE_PYPI_PASSWORD
and
engine_cmd_test
with CAPSULE_ENGINE_CMD_TEST
Example System Configuration File
Here's what an example system configuration file might look like:
scm_git_parent_path: /srv/myclonefolder
scm_github_api_endpoint: https://git.mycorpsubnet.example.com/v2
scm_github_web_endpoint: https://git.mycorpsubnet.example.com/v2
Step pre/post hooks and overrides
CapsuleCD is completely customizable, to the extent that you can run your own Ruby code as pre
and post
hooks before every step. To add a pre
/post
hook or override a step, just modify your config yml
file by
adding the step you want to modify, and specify pre
or post
as a subkey. Then specify your shell commands as a list
---
scm_init:
pre:
- echo "override pre_scm_configure"
- git clone ...
post: |
# do additional cleanup or anything else you want.
- echo "override post_scm_configure"
assemble_step:
post: |
# this post hook runs after the assemble_step runs
- echo "override post_build_step"
Testing
Test suite and continuous integration
CapsuleCD provides an extensive test-suite based on go test
and a full integration suite which uses go-vcr
.
You can run all the integration & unit tests with go test $(glide novendor)
CircleCI is used for continuous integration testing: https://circleci.com/gh/AnalogJ/capsulecd
Contributing
If you'd like to help improve CapsuleCD, follow the instructions in CONTRIBUTING.md
Note that if you would like to do development without Docker, you'll also need to ensure that you have the git2go
dependencies installed on your machine.
You can install these dependencies by using your system's package manager.
- openssl
- libgit2
- libssh2
Work your magic and then submit a pull request. We love pull requests!
If you find the documentation lacking, help us out and update this README.md. If you don't have the time to work on CapsuleCD, but found something we should know about, please submit an issue.
To-do ListWe're actively looking for pull requests in the following areas:
We use SemVer for versioning. For the versions available, see the tags on this repository.
AuthorsJason Kulatunga - Initial Development - @AnalogJ
LicenseCapsuleCD is licensed under the MIT License - see the LICENSE.md file for details
ReferencesTwice a month we will interview people behind open source businesses. We will talk about how they are building a business on top of open source projects.
We'll never share your email with anyone else.