Prove that G forms a group under matrix multiplication.

Matrix $A = \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix}$ represents a shear transformation.
The form $\begin{pmatrix} 1 & k \\ 0 & 1 \end{pmatrix}$ indicates a shear parallel to the x-axis with a shear factor of $k$.
Therefore, matrix $A$ represents a shear parallel to the x-axis with shear factor 2.

Matrix $B = \begin{pmatrix} 0 & 1 \\ -1 & 0 \end{pmatrix}$ represents a rotation transformation.
Comparing it to the general anti-clockwise rotation matrix $\begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}$:
$\cos\theta = 0$ and $\sin\theta = -1$. This corresponds to an angle $\theta = 270^\circ$ (or $\frac{3\pi}{2}$ radians) anti-clockwise.
Therefore, matrix $B$ represents a rotation of $270^\circ$ anti-clockwise about the origin (or $90^\circ$ clockwise about the origin).

a) (ii)

A point is transformed by matrix $A$ followed by matrix $B$. This means the transformation is $B(AX)$, so the composite matrix is $BA$.

The composite matrix of the transformations is:

a) (iii)

Assuming "point A" is a typo for "the point (2,3)" mentioned in part (ii), and "the transformation in (i)" refers to the composite transformation calculated in part (ii).

Let the point be $P = \begin{pmatrix} 2 \\ 3 \end{pmatrix}$. The image $P'$ is given by $BA \cdot P$.

The image of the point $(2,3)$ under the composite transformation is: