Skip to content

Commit

Permalink
Update course information for Summer Semester 2024 (#9)
Browse files Browse the repository at this point in the history
* Update general information

* Update lecture and homework headers

* Update homework due dates

* Add course timeline
  • Loading branch information
adrhill authored Mar 20, 2024
1 parent 4a1c558 commit 4abef68
Show file tree
Hide file tree
Showing 17 changed files with 152 additions and 120 deletions.
18 changes: 15 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ Follow the [installation instructions](https://adrhill.github.io/julia-ml-course
on the course website.

## Contents
The course is taught in five weekly sessions of three hours.
### Lectures
The **first half of the course** is taught in five weekly sessions of three hours.
In each session, two lectures are taught:

| Week | Lecture | Content |
Expand Down Expand Up @@ -80,9 +81,20 @@ The lectures and the homework cover the following packages:
| ProfileView.jl | 9 | Profiler |
| Cthulhu.jl | 9 | Type inference debugger |

### Project
In the **second half of the course**, after passing the homework,
students work in groups on a small programming project of their choice,
learning best practices for package development in Julia, such as:
* how to structure and develop a package
* how to write package tests
* how to write and host package documentation

During code review sessions, students give each other feedback on their projects
before presenting their work in end-of-semester presentations.

[site-url]: https://adrhill.github.io/julia-ml-course/
[ml-group-url]: https://www.tu.berlin/ml
[isis-url]: https://isis.tu-berlin.de/course/view.php?id=35533
[ml-group-url]: https://web.ml.tu-berlin.de
[isis-url]: https://isis.tu-berlin.de/course/view.php?id=37588

[goto-badge]: https://img.shields.io/badge/-Go%20to%20course%20website-informational
[isis-badge]: https://img.shields.io/badge/TU%20Berlin-ISIS%20page-red
4 changes: 2 additions & 2 deletions homework/H1_Basics.jl
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,13 @@ html"""
Homework 1: Julia Programming Basics
</p>
<p style="font-size: 20px;">
TU Berlin, Winter Semester 23/24<br>
TU Berlin, Summer Semester 2024<br>
</p>
</div>
"""

# ╔═╡ bdcb27c5-0603-49ac-b831-d78c558b31f0
md"Due date: **Monday, November 27th 2023 at 23:59**"
md"Due date: **Monday, April 29th 2024 at 23:59**"

# ╔═╡ 6be73c03-925c-4afa-bd66-aca90e6b49fe
md"This notebook gives you live feedback!
Expand Down
38 changes: 19 additions & 19 deletions homework/H2_Linear_Algebra.jl
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,13 @@ html"""
Homework 2: Indexing & Linear Algebra
</p>
<p style="font-size: 20px;">
TU Berlin, Winter Semester 23/24<br>
TU Berlin, Summer Semester 2024<br>
</p>
</div>
"""

# ╔═╡ bdcb27c5-0603-49ac-b831-d78c558b31f0
md"Due date: **Monday, December 4th 2023 at 23:59**"
md"Due date: **Monday, May 6th 2024 at 23:59**"

# ╔═╡ ddd6e83e-5a0d-4ff0-afe4-dedfc860994c
md"### Student information"
Expand Down Expand Up @@ -233,7 +233,7 @@ task(
```
If the image contains an uneven number of columns, ignore the center column.
For example, applying `my_crop` to an input matrix
For example, applying `my_crop` to an input matrix
```julia
3×5 Matrix{Int64}:
1 4 7 10 13
Expand Down Expand Up @@ -305,14 +305,14 @@ my_crop(img)

# ╔═╡ 3e933ccb-a6f3-4c2c-99ae-7ff03ed3a786
md"## Exercise 2: Singular value decomposition
The [Singular value decomposition](https://en.wikipedia.org/wiki/Singular_value_decomposition) (SVD) factorizes matrices $M$ into
The [Singular value decomposition](https://en.wikipedia.org/wiki/Singular_value_decomposition) (SVD) factorizes matrices $M$ into
$M = U \Sigma V' \quad .$
LinearAlgebra.jl implements SVD in the function `svd`. For this implementation of SVD:
LinearAlgebra.jl implements SVD in the function `svd`. For this implementation of SVD:
* $\Sigma$ is a vector of sorted, non-negative real numbers, the [singular values](https://en.wikipedia.org/wiki/Singular_value)
* $U$ is a unitary matrix
* $V$ is a unitary matrix
* $V$ is a unitary matrix
* $V'$ is the adjoint / conjugate transpose of $V$
"

Expand Down Expand Up @@ -400,7 +400,7 @@ hint(
@bind n Slider(1:50, default=50, show_value=true)

# ╔═╡ 8401a066-f5c0-4d86-8d0e-e9c287762252
md"Note that from a numerical perspective, it is a bit unusual to multiply the factorization $U \tilde{\Sigma} V'$ back together into an array $\tilde{A}$. We did this so we can apply the low-rank approximation to our image.
md"Note that from a numerical perspective, it is a bit unusual to multiply the factorization $U \tilde{\Sigma} V'$ back together into an array $\tilde{A}$. We did this so we can apply the low-rank approximation to our image.
Let's compute a rank $n=$ $(n) approximation. Adjust the rank using the following slider:"

Expand Down Expand Up @@ -436,7 +436,7 @@ $k(x_i, x_j) = \exp\left(-\frac{||x_i - x_j|| ^2}{2\sigma^2}\right)$
)

# ╔═╡ 7f9afd24-fd1c-49cd-b6bf-598cea9758c7
md"Note that you can type σ using `\sigma<TAB>`.
md"Note that you can type σ using `\sigma<TAB>`.
`σ`'s default value `σ_slider` is defined at the bottom of this notebook."

Expand Down Expand Up @@ -592,7 +592,7 @@ A \ b # Approach B: left division operator
C \ b # Approach C: left division operator and Cholesky factorization

# ╔═╡ cb3a2a70-7a4c-4adb-9983-95614645971b
md"In this example, all three approaches compute the correct result. However, matrix inversion is often numerically unstable, which is why it should be avoided.
md"In this example, all three approaches compute the correct result. However, matrix inversion is often numerically unstable, which is why it should be avoided.
The approach using the Cholesky factorization is numerically stable, fast and therefore well suited for the next exercise:"

Expand All @@ -613,14 +613,14 @@ task(
md"Implement Kernel Ridge Regression in the function `kernel_ridge` below.
##### Step 1
Inside the function `kernel_ridge`, construct the training kernel matrix $K_{XX}$ using the function `kernel`. Add a regularization term $\lambda I$, where where $I$ is the identity matrix and $\lambda \in \mathbb{R}$ is a regularization term:
Inside the function `kernel_ridge`, construct the training kernel matrix $K_{XX}$ using the function `kernel`. Add a regularization term $\lambda I$, where where $I$ is the identity matrix and $\lambda \in \mathbb{R}$ is a regularization term:
$\tilde{K}_{XX} = K_{XX} + \lambda I$
Don't specify the kernel bandwidth $\sigma$.
##### Step 2
Use the Cholesky decomposition on $\tilde{K}_{XX}$ to compute
Use the Cholesky decomposition on $\tilde{K}_{XX}$ to compute
$\alpha = (K_{XX} + \lambda I)^{-1}y \quad .$
Expand Down Expand Up @@ -693,7 +693,7 @@ else
elseif result isa AbstractArray
keep_working(
md"""
The `predict` function should return a scalar prediction $\hat{y}$ for scalar inputs $\hat{x}$.
The `predict` function should return a scalar prediction $\hat{y}$ for scalar inputs $\hat{x}$.
Currently, an array of size $(size(result)) is returned.
If the size is (1,) or (1, 1), you can use the function `only` on your output to access the only element in it. Alternatively, use the dot product.
Expand Down Expand Up @@ -739,12 +739,12 @@ Assume an input dataset $$X = [x_1, x_2, \ldots x_n] \in \mathbb{R}^{d\times n}$
* $n$ is the number of data points in $X$
Substracting the mean over all data points gives us the *mean-centered* data matrix $\hat{X}$.
**Idea:** PCA with $k$ principal components finds the projection $W_k = [w_1, w_2, \ldots, w_k]$ that maximizes the variance in the projected data $W_k^T\hat{X}$.
**Idea:** PCA with $k$ principal components finds the projection $W_k = [w_1, w_2, \ldots, w_k]$ that maximizes the variance in the projected data $W_k^T\hat{X}$.
![PCA Plot](https://i.imgur.com/FEyXwwX.png)
In the example above, this maximization of variance would correspond to a projection $W_1^T\hat{X} = w_1^T\hat{X}$ of all datapoints onto the red arrow, reducing the dimensionality from 2D to 1D.
In the example above, this maximization of variance would correspond to a projection $W_1^T\hat{X} = w_1^T\hat{X}$ of all datapoints onto the red arrow, reducing the dimensionality from 2D to 1D.
"

# ╔═╡ 067c7894-2e3f-43de-998f-4f1a250bc0f7
Expand Down Expand Up @@ -778,9 +778,9 @@ Compute the sample covariance matrix $C = \hat{X}\hat{X}^T$.
##### Step 3
Compute an eigendecomposition of $C$ into eigenvectors $w_i$ and eigenvalues $\lambda_i$
using the function `eigen`.
using the function `eigen`.
Select the eigenvectors $w_i$ corresponding to the $k$ largest eigenvalues $\lambda_i$,
such that $\lambda_1\ge\lambda_2\ge\ldots\ge\lambda_k$:
Expand Down Expand Up @@ -815,7 +815,7 @@ md"### Exercise 4.2 – Visualization"
task(
md"
Try to replicate the following plot as on the dataset `X_test` as close as you can.
![PCA Plot](https://i.imgur.com/FEyXwwX.png)
Expand Down
8 changes: 4 additions & 4 deletions homework/H3_Custom_Types.jl
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,13 @@ html"""
Homework 3: Custom types
</p>
<p style="font-size: 20px;">
TU Berlin, Winter Semester 23/24<br>
TU Berlin, Summer Semester 2024<br>
</p>
</div>
"""

# ╔═╡ bdcb27c5-0603-49ac-b831-d78c558b31f0
md"Due date: **Monday, December 11th 2023 at 23:59**"
md"Due date: **Monday, May 13th 2024 at 23:59**"

# ╔═╡ ddd6e83e-5a0d-4ff0-afe4-dedfc860994c
md"### Student information"
Expand Down Expand Up @@ -685,8 +685,8 @@ Markdown.MD(
"Task",
"Optional task (3 extra points)",
[
md"Implement the *Adam* optimizer, which was introduced
in [this paper](https://arxiv.org/abs/1412.6980) by Kingma and Ba.
md"Implement the *Adam* optimizer, which was introduced
in [this paper](https://arxiv.org/abs/1412.6980) by Kingma and Ba.
Add it to the visualization above and compare it to Optimizer.jl's Adam implementation.",
],
),
Expand Down
12 changes: 6 additions & 6 deletions homework/H4_Deep_Learning.jl
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,13 @@ html"""
Homework 4: Deep Learning
</p>
<p style="font-size: 20px;">
TU Berlin, Winter Semester 23/24<br>
TU Berlin, Summer Semester 2024<br>
</p>
</div>
"""

# ╔═╡ ce8fd3c2-4fe8-4408-8479-efc81f1a4147
md"Due date: **Monday, December 18th 2023 at 23:59**"
md"Due date: **Monday, May 20th 2024 at 23:59**"

# ╔═╡ 42a8576f-d0ab-4915-864f-c1c2fd65e4e4
md"### Student information"
Expand Down Expand Up @@ -178,7 +178,7 @@ convert2image(MNIST, x), y

# ╔═╡ efb62b97-c6fc-495f-a0ae-bf0c2e5485d3
md"#### Preprocessing
We preprocess the data for use with Flux.jl by one-hot encoding the targets `y`
We preprocess the data for use with Flux.jl by one-hot encoding the targets `y`
and reshaping the features `x` to WHCN (width, height, color channels, batch size) format:"

# ╔═╡ a173003f-9b32-4042-a263-b5af35106587
Expand Down Expand Up @@ -209,7 +209,7 @@ md"Run training: $(@bind run_training CheckBox(default=false))"
# ╔═╡ fe22ab4f-6119-4397-ac25-ce840a176777
if run_training # Do NOT modify this!

# Write your code here
# Write your code here
end

# ╔═╡ ae26551a-d4dd-4b37-a02d-a089de359f70
Expand Down Expand Up @@ -333,8 +333,8 @@ task(
# ╔═╡ 0b11ef46-0c80-4255-bb22-23281704916f
task(
md"""
Adapt the training loop from *Lecture 7*
to train the LeNet-5 model from exercise 1.2 (with `MyDense` layers)
Adapt the training loop from *Lecture 7*
to train the LeNet-5 model from exercise 1.2 (with `MyDense` layers)
on the Fashion-MNIST dataset.
1. Use the `Flux.logitcrossentropy` loss function
2. Use a suitable gradient-based optimizer, e.g. `Descent` or `Adam`
Expand Down
6 changes: 3 additions & 3 deletions lectures/E1_Installation.jl
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ html"""
Adrian Hill
</p>
<p style="font-size: 20px;">
TU Berlin, Winter Semester 23/24
TU Berlin, Summer Semester 2024
</p>
</div>
"""
Expand Down Expand Up @@ -188,11 +188,11 @@ You can then copy this URL into the *"Open a notebook"* field of the Pluto start
![Starting page](https://i.imgur.com/o7TjYX5.png)
Doing this will ensure that you are working with the latest version of the lectures and homework.
Doing this will ensure that you are working with the latest version of the lectures and homework.
"""

# ╔═╡ f1d4d38b-5593-4493-976a-e4b9d486007a
tip(md"If you are familiar with Git, you can also clone the repository of this course.
tip(md"If you are familiar with Git, you can also clone the repository of this course.
Just make sure to regularly `git pull` to keep your copy of the course up to date.")

Expand Down
2 changes: 1 addition & 1 deletion lectures/E2_Help.jl
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ html"""
Adrian Hill
</p>
<p style="font-size: 20px;">
TU Berlin, Winter Semester 23/24
TU Berlin, Summer Semester 2024
</p>
</div>
"""
Expand Down
4 changes: 2 additions & 2 deletions lectures/L1_Basics_1.jl
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ html"""
Adrian Hill
</p>
<p style="font-size: 20px;">
TU Berlin, Winter Semester 23/24
TU Berlin, Summer Semester 2024
</p>
</div>
"""
Expand Down Expand Up @@ -335,7 +335,7 @@ typeof(π)

# ╔═╡ b41bee43-f510-483f-9a12-d9d577367272
md"### Rational numbers
For integers $p$ and $q$,
For integers $p$ and $q$,
rational numbers $\frac{p}{q}$ can be expressed as fractions using the syntax `p//q`.
"

Expand Down
22 changes: 11 additions & 11 deletions lectures/L2_Basics_2.jl
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ html"""
Adrian Hill
</p>
<p style="font-size: 20px;">
TU Berlin, Winter Semester 23/24
TU Berlin, Summer Semester 2024
</p>
</div>
"""
Expand Down Expand Up @@ -156,7 +156,7 @@ v7 = [sqrt(x) for x in 1:10 if x % 2 == 0]

# ╔═╡ c66a192f-8bd7-462d-a062-cb670b07b723
md"""## Indexing
We can index into vectors using square bracket notation.
We can index into vectors using square bracket notation.
Let's create a small test vector with values from 1 to 10 for this purpose:
"""
Expand Down Expand Up @@ -244,8 +244,8 @@ end
# ╔═╡ 43c078fb-7964-48e0-92fd-55d306f36b1d
#! format: off
tip(
md"If a method requires a lot of calls to `@view`,
you can also use the `@views` macro (with an extra `s`) on a function, loop, or `begin ... end` block
md"If a method requires a lot of calls to `@view`,
you can also use the `@views` macro (with an extra `s`) on a function, loop, or `begin ... end` block
to turn every indexing operation inside of it into a view!
**Example:**
Expand All @@ -254,7 +254,7 @@ The previous code block could have been written as
```julia
@views begin # every indexing operation in this code block will create a view
x2 = collect(1:10)
y2 = x2[2:4] # creates view onto x2 due to @views
y2 = x2[2:4] # creates view onto x2 due to @views
y2[1] = 42
end
```
Expand Down Expand Up @@ -312,8 +312,8 @@ typeof(t1)
# ╔═╡ 79c55150-0457-49b7-90a5-2cad980b2cb9
tip(
md"""
For performance reasons, it is best to avoid Vectors of type `Vector{Any}`: Julia's compiler can't infer element types and therefore also can't specialize code, resulting in bad performance.
For performance reasons, it is best to avoid Vectors of type `Vector{Any}`: Julia's compiler can't infer element types and therefore also can't specialize code, resulting in bad performance.
You can find out more about type promotion in the [Julia documentation on Conversion and Promotion](https://docs.julialang.org/en/v1/manual/conversion-and-promotion/#conversion-and-promotion).
""",
)
Expand Down Expand Up @@ -856,7 +856,7 @@ end

# ╔═╡ b857984b-ec1b-4409-bdf0-438228950f39
md"""# Broadcasting on arrays
We've already seen broadcasting on vectors in the previous lecture.
We've already seen broadcasting on vectors in the previous lecture.
For higher-dimensional arrays, the behaviour is [a bit more complicated]((https://docs.julialang.org/en/v1/manual/arrays/#Broadcasting)):
> Broadcast **expands singleton dimensions in array arguments to match the corresponding dimension in the other array** without using extra memory, and applies the given function elementwise
Expand All @@ -868,9 +868,9 @@ P &\in \mathbb{R}^{2×3×1×1×1} \\
Q &\in \mathbb{R}^{1×1×4×5} \quad .
\end{align}$
Julia will expand $P$ and $Q$ to
Julia will expand $P$ and $Q$ to
$\tilde{P},\,\tilde{Q} \in \mathbb{R}^{2×3×4×5×1}$
$\tilde{P},\,\tilde{Q} \in \mathbb{R}^{2×3×4×5×1}$
without allocating extra memory, then add them:
Expand Down Expand Up @@ -900,7 +900,7 @@ size(R)

# ╔═╡ b2cdd7d0-e009-4fca-a16b-ddd19cabc6a0
md"""# Further resources
In this lecture, we didn't get to cover Cartesian Indices, which are a great tool for implementing algorithms on high-dimensional arrays.
In this lecture, we didn't get to cover Cartesian Indices, which are a great tool for implementing algorithms on high-dimensional arrays.
The following blogpost gives a great introduction:
- [Multidimensional algorithms and iteration](https://julialang.org/blog/2016/02/iteration/) by Tim Holy
Expand Down
Loading

0 comments on commit 4abef68

Please sign in to comment.