Define machine learning, explain the difference between supervised and unsupervised learning, and state some future directions in AI research.

Here are the answers to the questions from the exam paper:

QUESTION 3

a) Using AI knowledge define machine learning?
Machine learning is a subset of artificial intelligence that enables systems to learn from data, identify patterns, and make decisions or predictions with minimal human intervention. It allows computers to improve their performance on a specific task over time through experience.

b) Explain the difference between supervised and unsupervised learning.

• Supervised Learning: This type of learning uses labeled data, meaning the input data comes with corresponding correct output labels. The model learns a mapping function from the input to the output, aiming to predict the output for new, unseen inputs. Examples include classification (e.g., spam detection) and regression (e.g., predicting house prices).
• Unsupervised Learning: This type of learning deals with unlabeled data, where there are no predefined output labels. The model's goal is to discover hidden patterns, structures, or relationships within the data on its own. Examples include clustering (e.g., grouping customers by behavior) and dimensionality reduction.

c) State some future directions in AI research?
Future directions in AI research include Explainable AI (XAI) to make AI decisions transparent, Ethical AI to ensure fairness and accountability, advancements in Artificial General Intelligence (AGI) for human-level cognitive abilities, Federated Learning for privacy-preserving model training, and AI for scientific discovery to accelerate research in various fields.

QUESTION 4

a) What is deep learning?
Deep learning is a specialized subfield of machine learning that uses neural networks with multiple layers (deep architectures) to learn complex patterns and representations from large amounts of data. It excels at tasks involving images, speech, and text by automatically extracting hierarchical features.

b) Explain the structure of a typical neural network.
A typical neural network consists of an input layer, one or more hidden layers, and an output layer. Each layer contains multiple interconnected neurons (nodes). Neurons in one layer are connected to neurons in the next layer by weighted edges. Each neuron processes its inputs, applies an activation function, and passes the result to the next layer. The input layer receives raw data, hidden layers perform computations, and the output layer produces the final prediction or classification.

c) What is backpropagation?
Backpropagation is a fundamental algorithm used to train artificial neural networks. It works by calculating the gradient of the loss function with respect to the network's weights. This is done by propagating the error (the difference between the network's output and the true value) backward through the network, layer by layer. The calculated gradients are then used to adjust the weights of the connections in the network, typically using an optimization algorithm like gradient descent, to minimize the loss.

QUESTION 5

a) With examples explain reinforcement learning?
Reinforcement learning (RL) is a type of machine learning where an agent learns to make decisions by performing actions in an environment to maximize a cumulative reward. The agent learns through trial and error, receiving positive rewards for desirable actions and penalties for undesirable ones, without explicit programming for every scenario.

• Example 1 (Game Playing): In games like Chess or Go (e.g., AlphaGo), the agent learns to make moves by playing against itself or other players. A win results in a positive reward, and a loss results in a negative reward. Over many games, the agent learns optimal strategies.
• Example 2 (Robotics): A robotic arm can learn to pick up an object. It receives a reward for successfully grasping the object and a penalty for dropping it or missing. Through repeated attempts, it refines its motor control and grasping technique.

b) Using programming perspective give the importance of AI in societies.
From a programming perspective, AI is crucial for societies because it enables the development of software systems that can automate complex tasks, process vast amounts of data, and provide intelligent insights. This leads to:

• Enhanced Automation: AI-powered programs automate repetitive and dangerous tasks in manufacturing, logistics, and customer service, freeing human workers for more creative roles.
• Data-Driven Decision Making: AI algorithms analyze big data to identify trends, predict outcomes, and inform strategic decisions in finance, healthcare, and urban planning.
• Personalized Experiences: AI programs create tailored recommendations (e.g., streaming services, e-commerce) and adaptive learning platforms, improving user engagement and educational outcomes.
• Problem Solving: AI provides computational solutions for complex societal challenges, such as drug discovery, climate modeling, and smart city management, by processing and understanding intricate data patterns.

QUESTION 6

a) Explain the concept of word embeddings.
Word embeddings are numerical representations of words in a vector space, typically as dense vectors of real numbers. The key idea is that words with similar meanings or that appear in similar contexts are mapped to nearby points in this vector space. These embeddings are learned from large text corpora and capture semantic relationships, allowing algorithms to understand the meaning and context of words more effectively than traditional one-hot encodings.

b) What are some challenges in machine translation? (8 marks)
Machine translation faces several significant challenges:

• Ambiguity: Words can have multiple meanings (polysemy) depending on context, which is difficult for machines to resolve accurately.
• Idiomatic Expressions: Phrases whose meaning cannot be deduced from the individual words (e.g., "kick the bucket") are hard to translate literally.
• Syntactic and Morphological Differences: Languages have different grammatical structures, word orders, and ways of forming words, making direct translation complex.
• Cultural Nuances: Translating cultural references, humor, or specific social contexts accurately is very challenging.
• Low-Resource Languages: Many languages lack sufficient digital text data for training robust machine translation models.
• Domain-Specific Terminology: Translating specialized jargon in fields like medicine or law requires deep contextual understanding.

c) What is generative adversarial networks (GANs), and how do they work?
Generative Adversarial Networks (GANs) are a class of artificial intelligence algorithms used in unsupervised learning, primarily for generating new data instances that resemble the training data. A GAN consists of two neural networks, a generator and a discriminator, that compete against each other in a zero-sum game.

• How they work:
  1. The generator network takes random noise as input and tries to produce synthetic data (e.g., images) that look real.
  2. The discriminator network takes both real data from the training set and fake data from the generator as input. Its task is to distinguish between real and fake data.
  3. During training, the generator tries to fool the discriminator by creating increasingly realistic data, while the discriminator tries to get better at identifying fake data. This adversarial process continues until the generator produces data that the discriminator can no longer reliably distinguish from real data, resulting in highly realistic generated outputs.

QUESTION 7

a) What are convolutional neural networks?
Convolutional Neural Networks (CNNs) are a specialized type of neural network primarily designed for processing and analyzing grid-like data, such as images, videos, and sometimes audio. They are particularly effective at automatically learning spatial hierarchies of features from raw input data. CNNs achieve this through the use of convolutional layers, which apply filters to detect patterns like edges, textures, and shapes, followed by pooling layers for dimensionality reduction, and finally fully connected layers for classification or regression.

b) Explain the concept of overfitting in machine learning.
Overfitting occurs in machine learning when a model learns the training data too well, including its noise and specific details, to the extent that it performs poorly on new, unseen data. An overfit model has high accuracy on the training set but low accuracy on the test set because it has essentially memorized the training examples rather than learning generalizable patterns. This often happens when the model is too complex for the amount of training data available, leading it to capture random fluctuations instead of underlying relationships.

c) What is natural language processing (NLP), and what are some common NLP tasks?
Natural Language Processing (NLP) is a field of artificial intelligence that focuses on enabling computers to understand, interpret, and generate human language in a valuable way. It involves the interaction between computers and human (natural) languages.
Some common NLP tasks include:

• Sentiment Analysis: Determining the emotional tone or opinion expressed in a piece of text (e.g., positive, negative, neutral).
• Machine Translation: Automatically translating text or speech from one natural language to another.
• Text Summarization: Generating a concise and coherent summary of a longer text document.
• Named Entity Recognition (NER): Identifying and classifying named entities in text into predefined categories such as person names, organizations, locations, dates, etc.
• Chatbots and Virtual Assistants: Developing conversational AI systems that can understand user queries and provide relevant responses.