Heading into the second half of 2020, one of my professional goals is to improve my software development expertise. I want to implement best practices in my projects and continue learning about new approaches.

I think there are several things I'm doing well such as:

• Well structured project READMEs with
  • Project overview
  • Environment setup / Requirements
  • Example script snippets to run programs
  • Explanation of data and program structures
• Documentation
  • Filestrings, docstrings
  • Using comments to help explain why design choices were made
  • Precisely named variables
• Code structure
  • Plaintext (YAML) configuration
  • Argparse into a program runner
  • Python definitions
  • Shared utility functions
• Git usage
  • Pull requests
  • Branching, Git flow
  • Gitignore

There are also several areas I'm looking to improve:

• CLI tooling such as Click instead of argparse
• Dependency management
• Have used conda, pip and some venv
• Move to using poetry, docker and dockercompose
• Logging, learn proper usage
• Code linting i.e Black
• Global configs and environment setting

Beyond the above, there other areas I'm looking to learn such as:

• Catching up on the latest with Flask
• Airflow
• Pyspark
• Spinning up infrastructure EC2, EMR, S3
• Git advanced usage
• CI and testing (Great Expectations)

How I will improve expertise

The best way to improve expertise in these areas is with practice. Ideally I would have work situations where I can focus on implementing one new skill at a time.

When work is quiet, I will instead focus on learning these skills and applying them to my personal projects. There are a few personal projects I have been hacking around with recently including:

• Wow_auctions - An application that analyses game data from World of Warcraft to inform pricing policies for auctioning items
• personal_site - This website
• Otsukare - A website developed during masters degree to help users study Japanese language learning

I will be creating and releasing some further articles over the next few days to elaborate on the above plans further.