Machine Learning

460-4139/01 | 460-4139/02 | Winter 2025/26

Course Overview

This course delves deeper into machine learning techniques, focusing on advanced topics such as advance classification models, clustering models and neural networks including basic concepts of deep learning. Students will engage in hands-on projects to apply these concepts in real-world scenarios.

Lectures

📚

Lecture 0: Course Introduction

Course content and its evaluation.
List of recommended sources.

This introductory lecture provides an overview of the course structure, content, and evaluation methods. It also includes a list of recommended textbooks and online resources to support student learning throughout the course.

Learning Objectives

  • How to pass the course
  • Whats included in this course
  • Recommended literature and online resources
  • Understanding the course structure and expectations
📄 Download Presentation
📚

Lecture 1: Association Pattern Mining

Extracting patterns from data.
Efficient algorithms design.

This lecture introduces association pattern mining techniques for discovering relationships between items in large datasets, covering fundamental concepts like frequent itemsets, support and confidence measures, and the Apriori algorithm.

Learning Objectives

  • Understand the concepts of support and confidence rule mining.
  • Distinguish between meaningful patterns and random correlations.
  • Understand the Apriori algorithm and its efficiency.
  • Evaluate real-world applications of pattern mining.
📄 Download Presentation
📚

Lecture 2: Exploratory analysis

Understandiong the data
Efficient visualization of features

This lecture focuses on exploratory data analysis (EDA) techniques, emphasizing the importance of understanding data distributions, relationships between variables, and effective visualization methods to uncover insights and inform subsequent modeling decisions.

Learning Objectives

  • Understanding different data types and their characteristics.
  • Distinguish between meaningful patterns and random correlations.
  • Understand the importance of data visualization in EDA.
  • Evaluate real-world applications of exploratory data analysis.
📄 Download Presentation
📚

Lecture 3: Clustering

Understanding the data clustering techniques
Interpretation of the created clusters.

This lecture focuses on clustering techniques, explanation of the similarity and distance in real world data, emphasizing the importance of understanding different clustering algorithms, their applications, and the interpretation of the results to uncover hidden patterns in data.

Learning Objectives

  • Understand different clustering algorithms and their characteristics.
  • Distinguish between various distance metrics used in clustering.
  • Understand the importance of cluster validation techniques.
  • Evaluate real-world applications of clustering methods.
📄 Download Presentation
📚

Lecture 4: Outlier analysis

Understanding the data outlier detection techniques
Importance of outliers in real world data.

This lecture focuses on outlier detection techniques, its methods, importance of outlier detection in real world data. Several algorithm will be discussed with their parameters and usage.

Learning Objectives

  • Understand different outlier detection algorithms and their characteristics.
  • Distinguish between various methods and their pros and cons.
  • Understand the importance of outlier detection techniques.
  • Evaluate real-world applications of outlier detection methods.
📄 Download Presentation
📚

Lecture 5: Dimension reduction

Understanding the dimension reduction techniques
Application for dimension reduction methods

This lecture focuses on the mehods for dimension reduction. Several algorithm will be discussed with their parameters and usage. Benefits and principles will be discussed in detail.

Learning Objectives

  • Understand different dimension reduction algorithms and their characteristics.
  • Different methods and their application area will be discussed.
  • Complexity and principles of them will be demonstrated.
  • Real-world applications of dimension reduction techniques will be explored.
📄 Download Presentation
📚

Lecture 6: Classification

Understanding classification techniques
Application of classification methods

This lecture focuses on the methods for decision trees, Support vector machines, feature selection, and dimension reduction. Several algorithms will be discussed with their parameters and usage. Benefits and principles will be discussed in detail.

Learning Objectives

  • Understand different classification algorithms and their characteristics.
  • Different methods and their application area will be discussed.
  • Complexity and principles of them will be demonstrated.
  • Real-world applications of classification techniques will be explored.
📄 Download Presentation
📚

Lecture 7: Neural networks

Understanding neural network architectures
Application of neural networks

This lecture focuses on neural networks architecture, its principles, convolutional networks, and recurrent networks. Each type will be discussed in detail with explanation of the principles of their usage, benefits, and complexity.

Learning Objectives

  • Understand different neural network architectures and their characteristics.
  • Different methods and their application area will be discussed.
  • Complexity and principles of them will be demonstrated.
  • Real-world applications of neural networks will be explored.
📄 Download Presentation
📚

Lecture 8: Regression

Understanding regression techniques
Application of regression analysis

This lecture focuses on regression techniques, their principles, and various types of regression models, but mainly on linear regression and its regularization. Each type will be discussed in detail with an explanation of the principles of their usage, benefits, and complexity.

Learning Objectives

  • Understand different regression techniques and their characteristics.
  • Different methods and their application area will be discussed.
  • Complexity and principles of them will be demonstrated.
  • Real-world applications of regression analysis will be explored.
📄 Download Presentation
📚

Lecture 9: Advanced Concepts

Understanding advanced machine learning concepts
Application of advanced techniques

This lecture focuses on advanced machine learning concepts, their principles, and various types of advanced models. Each type will be discussed in detail with an explanation of the principles of their usage, benefits, and complexity.

Learning Objectives

  • Understand some of the advanced concepts and their characteristics.
  • Different methods and their application area will be discussed.
  • Complexity and principles of them will be demonstrated.
  • Real-world applications of advanced techniques will be explored.
📄 Download Presentation
📚

Lecture 10: Ensemble methods

Understanding ensemble methods
Application of ensemble techniques

This lecture focuses on ensemble methods in machine learning, their principles, and various types of ensemble models. Each type will be discussed in detail with an explanation of the principles of their usage, benefits, and complexity.

Learning Objectives

  • Understand some of the ensemble methods and their characteristics.
  • Different methods and their application area will be discussed.
  • Complexity and principles of them will be demonstrated.
  • Real-world applications of ensemble techniques will be explored.
📄 Download Presentation
📚

Lecture 11: Forecasting

Understanding forecasting methods
Application of forecasting techniques

This lecture focuses on forecasting methods in machine learning, their principles, and various types of forecasting models. Each type will be discussed in detail with an explanation of the principles of their usage, benefits, and complexity.

Learning Objectives

  • Understand some of the forecasting methods and their characteristics.
  • Different concepts will be discussed.
  • Complexity and principles of them will be demonstrated.
  • Real-world applications of forecasting techniques will be explored.
📄 Download Presentation

Exercises

💻

Exercise 1: Association Pattern Mining

Hands-on pattern mining.
Implementation of permutation.

This exercise focuses on practical applications of association pattern mining, where students will implement algorithms to extract meaningful patterns from datasets and evaluate their effectiveness using real-world examples.

Exercise Content

  • Generate all combinations without repetition of length 3 from 6 possible ones.
  • On one of the test files (chess, connect, ...), generate frequent patterns and calculate Support.
  • From the generated frequent patterns, write down the rules and their Confidence.
💻

Exercise 2: Exploratory Data Analysis

Hands-on data exploration.
Usage of visualization techniques.

This exercise focuses on practical applications of exploratory data analysis, where students will extract meaningful insights from datasets and evaluate their effectiveness using real-world examples.

Exercise Content

  • Download a dataset from Kaggle or UCI Machine Learning Repository.
  • Explore the dataset and understand them with selected plots.
  • Apply data transformation techniques to prepare the data for analysis.
💻

Exercise 3: Clusterign implementation

Implementation of Hierarchical Clustering.
Validation of knowledge and skills.

This exercise focuses on implementation of hierarchical clustering techniques, where students will create their own implementation of agglomerative clustering using single and complete linkage methods, and visualize the results using scatter plots.

Exercise Content

  • Implement two agglomerative clustering approaches – single linkage and complete linkage – using Manhattan and Euclidean metrics derived from a distance matrix.
  • Stop criteria: either terminate when a predefined number of clusters is reached or carry out full clustering and then cut the dendrogram at the appropriate level to yield the desired number of clusters.
  • Visualization: present the resulting cluster assignments using a scatter plot to display the clustering structure.
💻

Exercise 4: Clustering usage

Application of clustering algorithms.
Validation of knowledge and skills.

This exercise focuses on application of clustering techniques on the selected dataset from the Kaggle or UCI repository. Students will apply different clustering algorithms, compare their results, and visualize the clustering structure using scatter plots.

Exercise Content

💻

Exercise 5: Dimension Reduction

Application of dimension reduction algorithms.
Validation of knowledge and skills.

The aim of the exercise is to test the possibilities of dimension reduction and verify the ability of methods to reduce noise in data. Use SVD decomposition and examine the error of matrix reconstruction. Another task is to use PCA and TSNE methods to create a 2D visualization of data.

Exercise Content

  • Check the datasets in this page for experimenting.
  • Apply SVD (complete) and Reduced SVD method to Bars Dataset and evaluate also data with noise.
  • Reconstruct the data with different reduction (2, 5, 10, 16 basis functions).
  • Try the NNMF method similarly.
  • Take the Mnist dataset and apply SVD recomposition and visualize it in vector space. Coloring take from Labels
  • Try to reconstruct the reduced data and visualize it.
  • Use PCA and t-SNE for further visualization of the Mnist dataset.
💻

Exercise 6: Decision Trees

Implementation of Decision Tree
Validation of understnading to theory and split criteria

The aim of the exercise is to test ability to implement decision trees - an elementary algorithm with single splitting criteria that is easy to understand and interpret. Students should be able to apply the concepts learned in the course to build a decision tree from scratch.

Exercise Content

  • Implement a decision tree from scratch.
  • Use a simple dataset to train the decision tree.
  • Visualize the decision tree structure (console output is enough).
  • Evaluate the performance of the decision tree.
💻

Exercise 7: Classification

Application of classification algorithms
Validation of knowledge and comparisons.

The aim of the exercise is to test various classification algorithms from Scikit-learn library. Students should be able to apply preprocessing when needed, and compare results of different methods. You may use the prepared template and data.

Exercise Content

  • Apply the classification algorithms from Scikit-learn.
  • Use classical datasets to train different models.
  • Preprocessing the data to gain maximum performance.
  • Evaluate the performance of the model.
📄 Jupyter 📄 Data (CSV)
💻

Exercise 8: Regression

Application of regression algorithms and feature selection
Validation of knowledge and comparisons.

The aim of the exercise is to test various regression algorithms from Scikit-learn library. Students should be able to apply preprocessing when needed, feature selection, and compare results of different methods. You may use the prepared template and data.

Exercise Content

  • Apply the regression algorithms from Scikit-learn.
  • Use classical dataset or presented dataset to train different models.
  • Preprocessing the data to gain maximum performance.
  • Evaluate the performance of the model.
📄 Jupyter 📄 Data (CSV)
💻

Exercise 9: Imbalanced Learning

Application of different approaches for imbalanced data
Validation of knowledge and comparisons.

The aim of the exercise is to test various approaches for handling imbalanced datasets using Scikit-learn library. Students should be able to apply resampling techniques, evaluate model performance, and compare results of different methods. You may use the prepared template and data.

Exercise Content

  • Apply the resampling techniques from Scikit-learn.
  • Use classical dataset or presented dataset to train different models.
  • Preprocessing the data to gain maximum performance.
  • Evaluate the performance of the models.
📄 Jupyter
💻

Exercise 10: Forecasting

Application of different approaches for time series forecasting
Validation of knowledge and comparisons.

The aim of the exercise is to test various approaches for time series forecasting using Scikit-learn library. Students should be able to apply different forecasting techniques, evaluate model performance, and compare results of different methods. You may use the prepared template and data.

Exercise Content

  • Load a proper dataset.
  • Prepare data for forecasting tasks.
  • Preprocessing the data to gain maximum performance.
  • Evaluate the performance of the models.
📄 Jupyter
💻

Exercise 11: Deployment of ML model

How to deploy a ML model for production
Necessary task to provide model result as a service.

The goal of this exercise is to create a machine learning model capable of predicting the survival of passengers on the Titanic 2.0 (a notional project based on the original data). For the model, you must create a simple web application that allows you to obtain predictions based on input data.
Train the Scikit-learn (sklearn) model and save it. In the web application, load the trained model and use it for predictions.

Exercise Content

  • The first part of this exercise requires you to develop a machine learning model for Titanic survival prediction by loading the Titanic dataset, performing necessary data preprocessing (without being obligated to use all records or attributes), and then training any classification model of your choosing.
  • For robustness, it is suggested to use tools from sklearn.pipeline or sklearn.compose to integrate the preprocessing steps and the model into a unified object.
  • Once the model is trained, it needs to be saved, which must then be serialized—for instance, using Pickle—to allow the trained model to be easily loaded and utilized later in the web application.
  • Create a web application/service for obtaining predictions. - One option is to use Flask to create a simple API that returns the model's prediction for given parameters in JSON format.
  • Test your web application with predictions for real or invented inputs (e.g., 2 real examples from the dataset and 1 invented):
    • {“Age”: 22, “Fare”: 7.25, “SibSp”: 1, “Parch”: 0, “Pclass”: 3, “Embarked”: “S”, “Sex”: “male”}
    • {“Age”: 38, “Fare”: 71.2833, “SibSp”: 1, “Parch”: 0, “Pclass”: 1, “Embarked”: “C”, “Sex”: “female”}
    • {“Age”: 18, “Fare”: 10, “SibSp”: 1, “Parch”: 2, “Pclass”: 2, “Embarked”: “S”, “Sex”: “male”}
📄 PDF

Datasets

🗂️

Datasets: Association Pattern Mining

Data for pattern mining.
Usable on Exercise 1.

The data is stored in DAT format, which is a plain text format that captures a list of purchased goods/presence of goods in a transaction on each line/transaction.

🗂️

Datasets: Clustering datasets

Data for clustering tasks.
Usable on Exercise 3 and later.

The data is stored in CSV format, which is a plain text format that captures the most relevant features for clustering tasks. Consists mainly of numbers in two or more dimensions

🗂️

Datasets: Dimension Reduction

Data for dimension reduction tasks.
Usable on Exercise 5 and later.

The data is stored in compresed (ZIP) CSV format, which is a plain text format that captures the most relevant features for dimension reduction tasks. Each row is a whole "picture" represented by a vector of pixel values. all images are of square shape

Available datasets

The dataset Bars contains images of bars with 8x8 pixels with different levels of noise. Mnist is in the shape of 28x28 pixels, with lables stored separatelly in Mnist labels dataset.

🗂️

Datasets: Classification

Data for classification tasks.
Usable on Exercise 6 and later.

The data is stored in CSV format, where the last columns is mainly the class of the data.

🗂️

Datasets: Regression

Data for regression tasks.
Usable on Exercise 8 and later.

The data is stored in CSV format, where the last columns is mainly the class of the data.

🎯 Course Projects

Three exciting projects to demonstrate your machine learning expertise. Each project is designed to challenge your understanding and application of ML concepts. Each project will push your skills to the next level!

Important Deadlines

Clustering Project: November 30, 2025 (23:59)
Classification & Implementation: January 11, 2026

🔵

Project 1: Data Clustering Analysis

Points: 15-30
Deadline: November 30, 2025 (23:59)

📋 Project Overview

Perform clustering analysis with description of discovered groups for your selected dataset. Please inform the instructor about your dataset selection via email.

📊 Required Components

  • Basic description of the analyzed dataset
  • Brief data exploration
  • Data preprocessing
  • Application of clustering methods
  • Description of discovered clusters
  • Optional: 2D visualization of clusters
  • Mandatory: Final summary of interesting findings

⚙️ Technical Requirements

🔬 Clustering Variations

Perform 3 different clustering experiments (combinations of different preprocessing and clustering methods)

🛠️ Methods

Use at least 2 different clustering methods

📝 Format

Jupyter Notebook with code and interpretation, or PDF report + source code

⏱️ Estimated Time

3-6 hours (varies based on your skills)

🧪 What to Experiment With

  • Preprocessing methods for analyzed data
  • Clustering method parameters
  • Attribute selection for clustering:
    • Complete dataset
    • Manual selection (based on dataset description)
    • Feature selection using SelectKBest with chi² test or others

⚠️ Dataset Selection Guidelines

Recommended size: 1000+ records and 10+ attributes

Excluded datasets:

  • From exercises and lectures: Titanic, Country Data, Red Wine Quality
  • From ZSU: Credit Card Data, Audi Used Car Listings, FIFA 2022, Social/Gender/Study Data from Secondary School Students, Wine Dataset for Clustering

Note: Consider suitability for clustering. Datasets with predominantly binary or categorical attributes will be more challenging.

🎯 Scoring Criteria

Good projects meeting necessary conditions will earn 15-30 points. For maximum points, demonstrate:

  • Detailed consideration of all steps: preprocessing, method selection with parameter tuning
  • Thorough description of discovered clusters
  • Analytical approach throughout the project
🔴

Project 2: Data Classification/Regression

Points: 15-25
Deadline: January 11, 2026 | Oral defense during exam period

📋 Project Overview

Solve a classification or regression task for your selected dataset. This is your second classification/regression project, so strive to elevate your work to a higher level!

📊 Required Components

  • Dataset description and attribute overview
  • Step-by-step description of your classification process:
    • Data preprocessing methods
    • Justification for classifier evaluation approach
  • Test at least 4 classification methods (mandatory: DecisionTree and RandomForest, choose 2 more)
  • For each classifier:
    • Focus on parameter tuning
    • Conduct experiments with different parameter settings
    • Describe parameter influence on performance (text, tables, graphs)
  • Mandatory: Final summary comparing achieved results

🔍 Key Focus Areas (Dataset Dependent)

  • Cross-validation
  • Optimal model parameter selection
  • Feature selection
  • Handling class imbalance
  • And more...

📝 Deliverables

📄 Format

Jupyter Notebook with code and interpretation, or PDF report + source code

⏱️ Estimated Time

3-6 hours (varies based on your skills)

🎤 Defense

Oral defense scheduled during exam period

⚠️ Dataset Selection Guidelines

Recommended size: 1000+ records and 10+ attributes

Sources: Kaggle, UCI Repository, and other internet sources

Excluded datasets:

  • From exercises and lectures: Titanic, Country Data, Red Wine Quality
  • From ZSU: Credit Card Customers, Adult, Heart Failure Prediction Dataset, KDD Cup 99, UNSW_NB15, Bank Marketing Data Set
🟢

Project 3: Algorithm Implementation

Points: 15-25
Deadline: January 11, 2026 | Oral defense during exam period

📋 Project Overview

Create your own implementation of a selected algorithm (e.g., from lectures) to demonstrate your understanding of the method. Report your chosen method to the instructor via email.

💻 Implementation Requirements

  • Use any programming language of your choice
  • Include a demonstration of your implementation with sample data (e.g., Iris or Titanic dataset)
  • Compare with library implementation when possible:
    • Compare achieved scores
    • Compare computation time
    • Identify inefficient parts of your implementation
    • Suggest potential optimizations

🎯 Suggested Algorithms

Choose from these algorithms or propose your own:

Random Forest Neural Network AdaBoost Naive Bayes DBSCAN CLARANS OPTICS BIRCH ... and more!

📝 Important Notes

  • Your implementation doesn't need to be more efficient than library versions
  • Focus on understanding the algorithm deeply
  • Be able to identify performance bottlenecks
  • Demonstrate awareness of potential improvements

📄 Deliverables

🎤 Defense

Oral defense scheduled during exam period - be prepared to explain your implementation in detail

📧 Notification

Email the instructor about your selected algorithm before starting

💡 General Tips for All Projects

  • Start early - don't wait until the deadline!
  • Document your thought process and decisions
  • Use visualizations to support your findings
  • Be analytical - explain WHY, not just WHAT
  • Ask questions if you're unsure about requirements
× Dataset preview