Notes

Next step:

- [DONE] Create and upload lambda functions (from s3 buckets for now)
- [DONE] Versioning lambda functions on Updates
- [DONE] Set appropriate permissions for invoking lambda functions from Kinesis


- [DONE] Research Kinesis
  - [DONE] How to publish to Kinesis
  - [DONE] How to invoke lambda functions from Kinesis

- [DONE] Parse YAML file
- [DONE] Parse command line (and give appropriate syntax error)
- [DONE] Logging of the program

- [DONE] Create a workflow
- [DONE] Test the workflow
- [DONE] Track workflow

- REST API
  - [DONE] POST /publish
  - [DONE] GET /track/workflows/<workflow_id>/executions/<execution_id>

- [DONE] Fix all TODOs
- [DONE] Test pub/sub
- [DONE] Complete examples
- Monitor Lambda functions


Using xFlow
===========
Set the aws credentials in ~/.aws/config and ~/.aws/credentials

Testing:
  nosetests -svx tests

Validating config:
  python xflow/__init__.py examples/wordcount/wordcount.yaml -v

Configuring:
  python xflow/__init__.py examples/wordcount/wordcount.yaml -c

Publishing:
  python xflow/__init__.py examples/wordcount/wordcount.yaml \
                    --log-level INFO \
                    -p FileUploaded '{"execution_id":"aaa", "message":"Testing"}'

Server Mode:
  Start server:
    python xflow/__init__.py examples/wordcount/wordcount.yaml --log-level INFO -s

  Publish:
    curl -XPOST localhost/publish -d '{"stream":"FileUploaded", "event":{"execution_id":"ccc", "message":"Test with ccc"}}'

  Track:
    curl -v localhost/track/workflows/compute_word_count/executions/ccc


Releasing to pypi
http://sherifsoliman.com/2016/09/30/Python-package-with-GitHub-PyPI/
====================================================================
- Do all your commits
- git tag <VERSION> -m "Adds <VERSION> tag for PyPI"
- git push --tags origin master
- python setup.py register -r pypitest
- python setup.py sdist upload -r pypitest
- python setup.py register -r pypi
- python setup.py sdist upload -r pypi