Neural Networks (COMP 258)

This course introduces artificial neural networks and deep learning models, emphasizing both theoretical foundations and practical implementation using modern machine learning frameworks.

Course Description

COMP 258 explores artificial neural networks and their application to real-world problems. Students study foundational neuron models, including McCulloch-Pitts, Hebbian learning, Perceptrons, ADALINE, and Multi-Layer Perceptrons (MLPs), before progressing to modern deep learning architectures.

Topics include convolutional neural networks (CNNs), sequence models such as recurrent neural networks (RNNs) and long short-term memory (LSTM) networks, attention mechanisms, and transformer-based models including Vision Transformers. Key learning components include backpropagation, activation functions, optimization methods, regularization techniques, and embedding layers.

Students gain hands-on experience using Keras and TensorFlow to design, train, and evaluate neural networks for tasks such as image recognition, natural language processing, and generative AI. AI-assisted development tools are introduced to support experimentation, debugging, and model refinement.

Course Learning Outcomes

Upon successful completion of this course, students will be able to:

Analyze and compare early artificial neuron models, including McCulloch-Pitts, Perceptron, and ADALINE.
Explain the mathematical foundations of neural networks, including weighted sums, activation functions, loss functions, and gradient-based learning.
Select appropriate neural network architectures for supervised and unsupervised learning problems.
Design and implement neural network models using Keras and TensorFlow to solve real-world classification and prediction tasks.
Apply regularization and optimization techniques to improve training stability and generalization.
Build and evaluate convolutional neural networks (CNNs) for image recognition tasks.
Apply recurrent neural networks (RNNs) and LSTM models to sequence modeling and natural language processing problems.
Apply attention-based transformer models, including Vision Transformers, and experiment with generative AI approaches for real-world applications.

Further Learning – Neural Networks and AI Frameworks

The following frameworks and research resources support continued learning and experimentation in neural networks and deep learning.

TensorFlow – Open-source framework for large-scale machine learning.
Keras – High-level neural networks API built on top of TensorFlow.
DeepLearning.AI – Courses and resources on modern deep learning.
OpenAI Research – Frontier research in generative models and AI systems.
Ollama – Tools for running large language models locally.

Course Materials

Primary Textbook

Géron, A. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow (3rd ed.). O’Reilly Media, 2023.

Reference Textbooks

Fausett, L. Fundamentals of Neural Networks. Pearson, 1994.
Rojas, R. Neural Networks: A Systematic Introduction. Springer, 1996 – Read online
Nielsen, M. Neural Networks and Deep Learning, 2019 – Read online
Goodfellow, I., Bengio, Y., Courville, A. Deep Learning. MIT Press, 2016 – Read online
Chollet, F., Watson M. Deep Learning with Python. Manning Publications, 2025 – Read online chapters
Loy, J. Neural Network Projects with Python. Packt, 2019.
Bonnin, R. Machine Learning for Developers. Packt, 2017.
Domingos, P. The Master Algorithm. Basic Books, 2015.