I won't tell the full mapping from poles(1)/zeroes(0) to the frequency response but I think I can explain the connection between frequency and zero/infinite response, why do you have infinite/zero response at $e^{-jw}=z_\text{zero/pole},$ i.e. what $e^{-jw}$ has to do with $z$.
The general form of the linear system is
$$y_n+a_1y_{n-1}+a_2y_{n-2}+\cdots = b_0 x_n + b_1x_{n-1}+b_2x_{n-2}+\cdots, $$ which can be solved in z-from as $$Y(z) = {(b_0 + b_1 z + b_2 z^2+\cdots)\over(1+a_1z+a_2z^2+\cdots)}X(z) = H(z)X(z) = {(1-z_0z)(1-z_1z)\cdots \over(1-p_0z)(1-p_1z)\cdots}X(z).$$
In the end, the series of binomial products $(1-z_0 z)\cdots{1\over 1-p_0 z}$ can be considered as a series of systems, where first output, is the input for another.
I would like to analyze the effect of single pole and zero. Let's single out the first zero, considering it the transfer function so that the rest of $H(z)X(z)$ is the input signal, $Y(z)=(1-z_0z)Χ(z),$ which corresponds to some $y_n = b_0x_n + b_1x_{n-1}.$ Let's take $b_0=b_1=1$ for simplicity. I mean that $y_n = x_n + x_{n-1}$.
What we want to determine the effect of the system H(z) upon harmonic signal. That is, the input is going to be test signal $$x_n = e^{jwn}\overset{z}{\leftrightarrow} 1 + e^{jw}z + e^{2jw} z^2 + \cdots = 1/(1-e^{jw})= X(z).$$
The response is going to be $$y_n = x_n + x_{n-1}|_{x_n = e^{jwn}} = e^{jwn} + e^{jw(n-1)} = e^{jwn}(1+e^{-jw})$$ that is, $1+e^{-jw}$ is the transfer function or $Y(z) = {(1+z)\over (1-e^{jw}z)} =(1+z)X(z)$.
Please note that $1+z$ basically says that output is sum of input signal plus shifted signal, since single $z$ stands for single clock delay in time domain.
Now, as explained in, $H(jw) = 1 + e^{-jw} = e^{-jw/2}(e^{jw/2}+e^{-jw/2}) =e^{-jw/2}2\cos(w/2)$. Cosine makes it to behave like low-pass filter
$$ \begin{cases}
w=0 & \Rightarrow &H(j0) = 1\cdot 2 \cos (0) = 2\\
w=\pi & \Rightarrow & H(j\pi) = e^{j\pi/2}2\cos(\pi/2) = 0
\end{cases}$$
It is also a good lesson that you get $2 cos \alpha = e^{i\alpha} + e^{-i\alpha}$ because you will supply the real signals rather than complex imaginary ones in real life.
LTI with impulse response = {1,-1} is $y_n=x_n -x_n|_{x_n=e^{jwn}} =e^{jwn}(1-e^{-jw})$ has transfer function of $H(jw) = (1-e^{-jw}) = e^{-jw/2}(e^{jw/2}-e^{-jw/2})=e^{-jw}2\sin(w/2)$, which has zero at $w=0$ since $sin(0)=0$ but it can be found from the frequency response $$H(jw)=1-e^{-jw} = 0 \Rightarrow e^{-jw} = 1 = e^0 \Rightarrow w = 0.$$
After the textbooks, I can spot the surprising coincidence between transfer function $H(z) = 1\pm z$ and frequency response $H(jw)=1\pm e^{-jw}$. That is, z somehow corresponds to $e^{-jw}$, which is important for zero/pole analysis. I read it like
sine z-factor stands for a clock shift and $y_n = x_n \pm x_{n-1}=0$ means that next sample is $\pm$ previous one to get zero response, we need to have $1\pm z=0$ in front of X(z). But, the frequency domain basis functions $e^{jwn}$ evolve by multiplying current value $e^{jw(n-1)}$ with $e^{jw}$ every clock. Therefore, we have $e^{jwn}(1 \pm e^{-jw}) =0$ as condition for constant zero output. The latter $1\pm e^{-jw}$ matches perfectly with zero transfer function $1\pm z=0$.
In general, single-zero LTI is given by $y_n = b_0 x_n + b_1 x_{n-1}$ or $$Y(z) = (b_0 + b_1 z)X(z) = (b_0+ b_1z)(1+x_1z+x_2z^2+\cdots) = b_0 + (b_0 x_1 + b_1 x_0)z + (b_0 x_2 + b_1 x_1)z^2 + \cdots. $$ When $b_0+b_1 z = 0$, i.e. when $z=-b_0/b_1,$ whereas frequency response is, $$y_n(x_n = e^{jwn}) = b_0 e^{jwn} + b_1 e^{jw(n-1)} = e^{jwn}(b_0+b_1 e^{-jw})= e^{jwn}b_0(1-z_0 e^{-jw}),$$
which goes to zero when $1-z_0 e^{-jw} = 0$ or $e^{-jw} = 1/z_0$, which matches the computation for $z$ if $z=e^{-jw}$. The only thing that bothers me is that fixed-amplitude complex exponential is not enough for the frequency (harmonic) basis. You cannot obtain arbitrary ratio $1/z_0= e^{-jw}$ by choosing appropriate frequency $w$, a decaying harmonic signal is needed for that. That is weird because I have heard that any signal can be represented as sum of (constant amplitude) sines and cosines. But, anyway, we see that system zero stands for relationship between adjacent samples of input signal. When they are right, the output is identically 0 and we can choose such such frequency $w$ so that zero $z = 1/z_0 = e^{-jw}.$
Now, what about the poles? Let's single out a single pole $a$. The system has a from of $y_n = a y_{n-1} + (x_n + x_{n-1} + \cdots)$, under assumption $y_0 = 0$, has z-transform of $Y(z) = X(z)/(1-az)$.
The feedback $a$ is equivalent to infinite impulse response ${1,a,a^2,\ldots} \overset{z}{\leftrightarrow} 1 + az + a^2z^2 + \cdots = 1/(1-az)$. It says that response is infinite when $z=1/a$. What does it mean if we apply the test signal $$x_n=e^{jwn}\overset{z}{\leftrightarrow} X(z) = 1+e^{jw}z+e^{2jw}z^2+\cdots = 1/(1-e^{jw}z)$$ to our system? We'll get $Y(z)={1\over 1-az}{1\over 1-e^{jw}z},$ or $$y_n = e^{jwn} + ae^{jw(n-1)} +a^2e^{jw(n-2)} +\cdots = e^{jwn}(1+ae^{-jw} + a^2e^{-2jw} + \cdots) ={e^{jwn}\over 1-ae^{-jw}}.$$ That is, frequency response is $1/(1-ae^{-jw}),$ which goes to infinity when $e^{-jw}=1/a,$ the same as $z_{pole}$ above, $e^{-jw}=z_{pole}=1/a$. But again, you can not always arrive at the pole $1/a$ adjusting the frequency $w$ alone. The frequency basis functions must be decaying amplitude in general and look like $(ke^{jw})^n$.
That is, zeroes or poles of the transfer function $H(z)$ happen to match the zeroes and poles of frequency response $H(jw)$, which is really amazing. I noticed that this is related to the relation between adjacent samples, $e^{jwn}/e^{jw(n-1)} = e^{jw} = 1/z_{zero}$ in case of zeroes. The fact that $e^{jwn}$ scales exponentially over time, along with the system with feedback $a$, also seems to be the key for matching between $e^{jw}$ and $z_{poles}$. It also seems important that you cannot simply look for the appropriate frequency of $e^{jwn}$, the basis function must also have adjustable amplitude factor $k^n$.
I would be happy if anybody could explain the same more condensely or more crisply.
