Sheet2graph Command-Line Program

We implement our own command in Python and distribute it via pip with pip install sheet2graph.

The command takes spreadsheets files (csv, xlsx) as input, and generates images (png, jpg, svg) based on the data contained.

Introduction, Repo Features And Environment

We explain how the project will be structured and what we are trying to accomplish.

• Settings up the project
• Configuring a virtual environment
• Making a simple command that outputs a string

First Graphs

We save out first graph from the spreadsheet data.

• Installing Plotly to generate graphs
• Pip freeze requirements
• Solving dependency problems
• Using Pandas to do simple data processing

Saving In A Folder

We will create a folder to save our image to.

• Saving the image in a folder programmatically
• Using Pathlib
• Add folder to .gitignore

First Commandline Options

We will add our first command line flags/options.

• Using argparse to parse commandline arguments and generate help
• Optional and required flags
• Add input file and graph type options
• Graph different graph types
• autogenerated help

Output Options

Here we add an option for different output locations.

• Using argparse to parse commandline arguments and generate help
• Combine options to output with a filename and with a folder
• Precendence of command flags

Output Format

We add Scalable Vector Graphics (SVG) output support.

• Add several output formats (svg, jpg, png) to our command

Generated Image Size

We will add output options to specify the size of the generated graphs.

• New option for output size
• Default and custom graph sizes

Refactoring And Type Checking

We start refactoring what we have until now, and add type annotations.

• Add basic annotations to functions
• Use Typeguard to enforce the annotations at runtime
• Annotations as a kind of documentation
• Add a new type of graph: the scatter graph

Reading Excel Files

We will add support for Excel files (.xlsx).

• add dependencies: Openpyxl and Xlrd for Excel support
• Fix type errors

Reading A File From Google Drive

In addition to a local file (.csv, .xlsx), we accept a Google Drive public document as input.

• Parsing the url address of the Google Drive document
• Adding the input option transparently for the user

Testing Helpers

Once we have the infrastructure for testing, we will write a few helpers to make the tests more concise and easier to write.

• Setup and teardown in a test suite
• Using Shutil to deal with filesystem operations
• Checking that a file is created in a specific path
• Checking the size of an image with PIL
• Running a command line application from a test

Print Data

After using hardcoded column names, we will start making the spreadsheet processing generic. This way our command will work with any spreadsheet file. The first step is to print the data of our input file, so the user can preview it.

• Print-only option to print the input file provided by the user
• Transforming the data with Pandas to index it by letter and 1-based integer (as in spreadsheet applications like Excel)
• Adding tests for the new indexing by letters and integer for columns and rows

Testing Data Selection

For each axis, we will allow the user to use expressions like 'b4,b5,b6,b7' or 'B4:B7' to select cells or ranges to graph.

• Add options '-x' and '-y' to select the data to be graphed
• Making the expressions case-insensitive
• Implementing a comma separated selection option
• Implementing a range selection option
• Adding tests first and making them pass after implementation, as in Test-Driven-Development (TDD)
• Better and more informative user messages in case of error
• Verifying and fixing problems in our Pandas implementation

Adding Labels To Graphs

We will check everything is working so far and add extra options to set the axis labels to a custom user-defined value.

• Debugging broken tests and making all tests pass after all our changes
• Adding an x label and y label options to our command, to specify the labels in the horizontal and vertical axis
• Debugging column types in pandas
• Using exceptions to deal with unreliable cases

Distributing Our Command

We will start preparing our command for distribution in pypi for it to be installable with pip, and testing the distribution in a test environment.

• Finding a proper name for our command
• Folder structure and necessary files
• Changes in the entry point of our program
• Generating a good README file in Markdown
• Examples and documentation

• Choosing a license
• Dependencies and versioning
• Setup.cfg and setup.py

Packaging And Uploading

We have all the files. We will test the distribution at test.pypi.org before publishing it in the real repository. This way we will be able to fix any mistakes.

• How to test in the test.pypi.org environment
• Using Twine to distribute your module
• Creating a new account at testing (test.pypi.org)
• Secure tokens in .pypirc or interactively for testing
• Versioning increases with each code change
• Fixing errors and testing the new version in the test environment
• Problems with images as documentation in the README file

Testing Install

After testing at test.pypi.org, we will fix an error with the example image in the documentation and distribute our command 'sheet2graph' to the real production environemnt at pypi.org.

• Fixing error with images in README.md at test.pypi.org
• Creating a new account at production (pypi.org)
• New secure tokens in .pypirc or interactively for production
• Solving problems with secure tokens

Uploading To Production

In this episode we put everything together and install our own command by typing 'pip install sheet2graph'. We test it both in a virtual environment and globally. We also talk about what this means to distribute code you develop easily to the world, something that now should be a lot more approachable. As always, we need to be mindful of publishing useful and tested code, and in general to play well within the ecosystem.

If you made it here I hoped you liked it. Subscribe or Connect with me on Twitter for updates on fromzerotofullstack.

• Solving problems with secure tokens
• Installing our new command using pip
• Testing on a new virtual environment
• Etiquette of publishing your modules and libraries

Resources for Python

A Programming language

Courses

Episodes

Short Videos