Given that A = represents a linear transformation T. (a) Show that the lines y = 2x and y = x are invariant under T. (b) The line 2x − y = 4 is transformed by T. Find the equation of its image.

Substitute $y = 2x$ into equations $(*)$ and $(**)$:

$$x' = 5x - 2(2x) = 5x - 4x = x$$ $$y' = 3x - (2x) = x$$

So, a point $(x, 2x)$ on the line $y=2x$ is mapped to $(x', y') = (x, x)$.
For the line $y=2x$ to be invariant, the transformed point $(x, x)$ must lie on the line $y=2x$.
This means $y' = 2x'$, so $x = 2x$. This implies $x=0$.
This shows that only the origin $(0,0)$ is an invariant point on the line $y=2x$. For any other point on $y=2x$, its image is not on $y=2x$. Therefore, the line $y=2x$ is not invariant under $T$ according to the standard definition of an invariant line (an eigenline).

Substitute $y = \frac{3}{4}x$ into equations $(*)$ and $(**)$:

$$x' = 5x - 2\left(\frac{3}{4}x\right) = 5x - \frac{3}{2}x = \frac{10x - 3x}{2} = \frac{7}{2}x$$ $$y' = 3x - \left(\frac{3}{4}x\right) = \frac{12x - 3x}{4} = \frac{9}{4}x$$

So, a point $(x, \frac{3}{4}x)$ on the line $y=\frac{3}{4}x$ is mapped to $(x', y') = \left(\frac{7}{2}x, \frac{9}{4}x\right)$.
For the line $y=\frac{3}{4}x$ to be invariant, the transformed point $\left(\frac{7}{2}x, \frac{9}{4}x\right)$ must lie on the line $y=\frac{3}{4}x$.
This means $y' = \frac{3}{4}x'$.
Let's check: $\frac{9}{4}x = \frac{3}{4}\left(\frac{7}{2}x\right)$

$$\frac{9}{4}x = \frac{21}{8}x$$

This implies $\frac{9}{4} = \frac{21}{8}$, or $18 = 21$, which is false.
Therefore, the line $y=\frac{3}{4}x$ is also not invariant under $T$ according to the standard definition.

Note: Based on the standard definition of an invariant line (a line whose points are mapped onto the line itself, often an eigenline), the given lines $y=2x$ and $y=\frac{3}{4}x$ are not invariant under the transformation $T$ represented by matrix $A$. The characteristic equation for invariant lines through the origin is $2m^2 - 6m + 3 = 0$, whose roots are $m = \frac{3 \pm \sqrt{3}}{2}$, which are not 2 or $\frac{3}{4}$. It is possible there is a misunderstanding of the question's intent or a typo in the problem statement.

We have the transformation equations:
$x' = 5x - 2y$
$y' = 3x - y$
We need to find the inverse transformation. First, calculate the determinant of $A$:

$$\det(A) = (5)(-1) - (-2)(3) = -5 + 6 = 1$$

The inverse matrix $A^{-1}$ is:

$$A^{-1} = \frac{1}{1} \begin{pmatrix} -1 & 2 \\ -3 & 5 \end{pmatrix} = \begin{pmatrix} -1 & 2 \\ -3 & 5 \end{pmatrix}$$

Now, we can find $x$ and $y$ in terms of $x'$ and $y'$:

$$\begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} -1 & 2 \\ -3 & 5 \end{pmatrix} \begin{pmatrix} x' \\ y' \end{pmatrix}$$ $$x = -x' + 2y'$$ $$y = -3x' + 5y'$$

$$2(-x' + 2y') - (-3x' + 5y') = 4$$ $$-2x' + 4y' + 3x' - 5y' = 4$$ $$(-2x' + 3x') + (4y' - 5y') = 4$$ $$x' - y' = 4$$

The equation of the image line is $x - y = 4$.

The equation of the image is $\boxed{x - y = 4}$.