MLOps tutorial for NOVA

Course material for MLOps summer course in Nova. Also available at this URL.

Learning outcomes of the curricular unit

1. Understand the fundamental principles of agile development
2. Understand how to integrate design, model development, training and operations in a continuous cycle.
3. Apply quality assurance concepts at every step of the process.
4. Create a complete model as well as a report in a software artifact that changes continuously.
5. Applying fundamental machine learning and scientific principles to the above design and reports.

Syllabus

Before the syllabus, we need to understand the principles | Web and read the introduction to the course | Web.

1. Design thinking for solving real-life problems | Web
2. Agile development | Web
3. Test driven development for scraping workflows | Web.
4. Containers for reproductible science | Web.

This will most probably not be included

1. Open science: creation and deployment of machine learning projects | Web.
2. MLOps tools.
3. Automation infrastructure using GitHub actions | Web.

Sessions

The course is organized as a bootcamp, with different sessions over which a project, and eventually a coauthored preprint, will be developed. Students can use this repository template to kick-start their projects with spell-checking workflows.

Updated data will be available in this repo; you can also explore data at this notebook.

Session objectives and material are as follows:

1. First session, meet and greet, organization and setup.
2. Second session, understanding agile.
3. Third session, downloading data.
4. Fourth session, extracting data, containerizing.
5. Fifth session, presenting information.

Some links

• MLOps page.

• MLOps stack.

• Twitter thread on academic vs. real-life data science/machine learning

• Data science workflow with R

• Fundamentals of data visualization

• Designing machine learning systems

• No-code ML

• MLOps zoomcamp

• Data science in the real world

• Splitting your data properly.

• Never process before splitting your data by @svpino

How to run this

Most tasks that are included in workflows are also in the top-level Makefile

Should be all of them, although at a certain point they can be out of sync.

But there are a few tools that you should have installed in your system

• Perl, preferably installed via perlbrew, including cpanm
• R
• Python

LICENSE

All text here is released under a cc-by-sa license, all code under the GPL.