{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# PyTorch Basics\n",
"\n",
"## Introduction to PyTorch\n",
"\n",
"### What is PyTorch?\n",
"\n",
"[PyTorch](https://pytorch.org/get-started/locally/) is a popular open-source deep learning framework known for its flexibility and ease of use. It's widely adopted in both research and industry for tasks ranging from simple machine learning models to complex neural networks.\n",
"\n",
"### Why PyTorch?\n",
"\n",
"- Dynamic computation graph: PyTorch's ability to dynamically build the computation graph at runtime makes it intuitive and easy to debug.\n",
"- Strong community support and integration with Python: PyTorch is Pythonic and integrates well with the Python data science stack.\n",
"- GPU acceleration: PyTorch makes it easy to move tensors to and from GPUs (supports Apple's Metal and Nvidia GPUs), which is crucial for training large models efficiently.\n",
"\n",
"## Tensors in PyTorch\n",
"\n",
"### What is a Tensor?\n",
"\n",
"Tensors are the fundamental data structures in PyTorch, similar to NumPy arrays but with the added capability of being used on a GPU.\n",
"\n",
"### Creating Tensors"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -q torch torchvision torchaudio"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"\n",
"# Creating a tensor from a list\n",
"tensor_a = torch.tensor([1.0, 2.0, 3.0])\n",
"print(tensor_a)\n",
"\n",
"# Creating a tensor with random values\n",
"tensor_b = torch.rand((2, 3)) # A 2x3 matrix of random numbers\n",
"print(tensor_b)\n",
"\n",
"# Creating a tensor with zeros\n",
"tensor_c = torch.zeros((3, 3)) # A 3x3 matrix of zeros\n",
"print(tensor_c)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Basic Tensor Operators"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Reshaping a tensor\n",
"tensor_reshaped = tensor_b.view(3, 2) # Reshape to 3x2\n",
"print(tensor_reshaped)\n",
"\n",
"# Tensor addition\n",
"tensor_sum = tensor_a + tensor_a\n",
"print(tensor_sum)\n",
"\n",
"# Indexing\n",
"print(tensor_a[1]) # Access the second element\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Moving a tensor to GPU\n",
"\n",
"# Check which device is available\n",
"available_device = (\n",
" \"cuda\"\n",
" if torch.cuda.is_available()\n",
" else \"mps\"\n",
" if torch.backends.mps.is_available()\n",
" else \"cpu\"\n",
")\n",
"\n",
"print(f\"Available device: {available_device}\")\n",
"\n",
"tensor_a_gpu = tensor_a.to(available_device)\n",
"print(tensor_a_gpu)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Autograd: Automatic Differentiation\n",
"\n",
"- PyTorch’s autograd system automatically calculates gradients, which are essential for training neural networks.\n",
"- Every operation on tensors keeps track of the computation history, allowing PyTorch to backpropagate errors automatically."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Create a tensor with gradient tracking enabled\n",
"x = torch.tensor(2.0, requires_grad=True)\n",
"\n",
"# Perform a computation\n",
"y = x ** 2 + 2* x ** 3\n",
"\n",
"# Backpropagate to compute the gradient\n",
"y.backward()\n",
"\n",
"# Print the gradient\n",
"print(x.grad) # Should output 28.0, the derivative of x^2 + 2x^3 at x=2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```{important}\n",
"This example shows how PyTorch automatically calculates the gradient of a tensor operation, which is essential for updating the weights during training.\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Dataset and DataLoaders\n",
"\n",
"### What is a Dataset in PyTorch?\n",
"\n",
"* Purpose: The `Dataset` class in PyTorch serves as an abstraction that allows you to manage, preprocess, and access your data in a consistent way.\n",
"\n",
"* Key Features:\n",
" - Handles how data is stored and accessed.\n",
" - Allows for custom data transformations and preprocessing.\n",
" - Integrates seamlessly with PyTorch’s `DataLoader` for efficient batching and shuffling.\n",
"\n",
"### What is a DataLoader in PyTorch?\n",
"\n",
"* Purpose: The `DataLoader` is an iterable that abstracts the complexity of handling data in batches, shuffling, and parallel loading.\n",
"\n",
"* Key Features:\n",
" - Efficiently loads data in mini-batches during training.\n",
" - Automatically shuffles data at the start of each epoch (if specified).\n",
" - Supports parallel data loading using multiple workers.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from torch.utils.data import Dataset\n",
"\n",
"class CustomDataset(Dataset):\n",
" def __init__(self, data, targets):\n",
" self.data = data\n",
" self.targets = targets\n",
"\n",
" def __len__(self):\n",
" # Return the total number of samples\n",
" return len(self.data)\n",
"\n",
" def __getitem__(self, idx):\n",
" # Retrieve the data sample and label at the specified index\n",
" sample = self.data[idx]\n",
" target = self.targets[idx]\n",
" return sample, target"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Explaining `__len__` and `__getitem__`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* __len__: Returns the total number of samples in your dataset. PyTorch uses this method to know how many iterations to run during training.\n",
"* __getitem__: Retrieves a specific sample from the dataset using its index. This method returns the data and its corresponding label, which PyTorch uses during training to form mini-batches."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"\n",
"# Generate random data: 6 samples, each with 2 features\n",
"torch.manual_seed(0) # For reproducibility\n",
"features = torch.rand(6, 2)\n",
"\n",
"# Generate random target values (e.g., for a regression problem)\n",
"targets = torch.rand(6, 1)\n",
"\n",
"print(f\"Features:\\n{features}\")\n",
"print(f\"\\nTarget:\\n{targets}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from torch.utils.data import DataLoader\n",
"\n",
"# Create an instance of the custom dataset\n",
"dataset = CustomDataset(data=features, targets=targets)\n",
"\n",
"# Create a DataLoader\n",
"data_loader = DataLoader(dataset, batch_size=2, shuffle=True)\n",
"\n",
"# Example of iterating through the DataLoader\n",
"for idx, (batch_data, batch_labels) in enumerate(data_loader):\n",
" print(f\"Batch {idx+1}:\\n========\")\n",
" print(f\"Data:\\n{batch_data}\")\n",
" print(f\"Targets:\\n{batch_labels}\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* `shuffle=True` ensures that the data is shuffled at the beginning of each epoch, which helps prevent the model from learning patterns based on the order of the data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"::::{dropdown} 🏋️ Exercise: What if we set `shuffle` to `False` and `batch_size` to 3?\n",
"::::{tip}\n",
"Check the Sample Size.\n",
"::::\n",
"::::"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}