8

The famous https://www.w3.org/TR/audio-eq-cookbook/ offers a set of [real] biquad filter calculation formulas that generally work fine.

However, when filter's frequency approaches Nyquist frequency the Q (bandwidth) specification of a filter becomes greatly distorted - usually it shrinks a lot (even though the author mentioned that he performed a necessary pre-warping).

I'm in quest for filter formulas that do not have such strong bandwidth distortion. I need peaking/bell, band-pass, low-pass, high-pass, high-shelf, notch filters. I know this can be done as previously I've bought a peaking/bell/band-pass filter formulas with less distortion, but they are still not perfect and I need other filter types.

So, I'm also willing to pay for the solution if the price is right.

Alternatively, if one could point me to an optimization algorithm that works with Z-domain filters that would be great, too. Unfortunately, most usual optimization algorithms do not work well in Z-domain - they can't optimize a set of parameters to match a desired frequency response (probably due to periodic functions used to calculate the frequency response).

robert bristow-johnson
  • 20,661
  • 4
  • 38
  • 76
aleksv
  • 181
  • 1
  • 5
  • Would FIR filters also do the job for you? Just asking because they're much easier to design, especially for more complicated specs. – Matt L. Nov 19 '14 at 16:42
  • Sorry, FIR filters are not suitable in this case. – aleksv Nov 19 '14 at 16:47
  • Complexity or delay? – Matt L. Nov 19 '14 at 16:49
  • They are not suitable due to delay, in real-time use. They are also too computationally-intensive. – aleksv Nov 19 '14 at 16:54
  • 1
    In order to effectively compensate the aliasing and warping artefacts for discrete filters you have to use very high orders. FIR would make the design much easier indeed, but there are also other methods. In order to give you a meaningful answer I would like to know what exactly your computational constraints are (including platform), and what your processing requirements are. Why do you need this behaviour, and would oversampling not suffice? – Jazzmaniac Nov 19 '14 at 19:21
  • This is to be used in music sequencer software, on many sound tracks at once. So, FIR and oversampling are not an option. High order IIR filters are not needed: as I've already noted, this is doable in biquad form, but needs complex math, that's the catch. The resulting response does not have to be an exact match to analog prototype at all frequencies, but should keep the bandwidth. – aleksv Nov 19 '14 at 19:29
  • 2
    well, there's a limit to what you can get with a 2nd-order IIR filter. there is another paper by Orfanidis that puts the gain at Nyquist to be something other than 0 dB (the Cookbook puts the Nyquist gain at 0 dB because that's how the bilinear transform maps it). for a 44.1 kHz sample rate, and 2nd-order biquad, i don't think you'll do much better than the Orfanidis design. – robert bristow-johnson Nov 19 '14 at 21:07
  • 1
    The best method I'm aware of is not directly mapping to the discrete domain but rather applying ODE solvers to the analog prototype. Specifically implicit solvers do a great job at preserving the frequency response. – Jazzmaniac Nov 19 '14 at 21:36
  • 1
    @Jazzmaniac: Why use an analog prototype instead of designing directly in the discrete-time domain? – Matt L. Nov 19 '14 at 22:10
  • @MattL.The analog prototype has desirable properties like scale invariance which makes the design of constant-Q filters specifically easy. But the real reason is that it avoids the artefacts introduced by a finite sampling rate like response aliasing and all kinds of response distortion. So you can design exactly what you want and using a ODE solver does not(!) necessarily map it to what we call z-domain. Higher order and/or implicit solvers map to a more general set of (recursive) discrete models and sometimes even violate strict linearity while still describing a linear analog system. – Jazzmaniac Nov 19 '14 at 22:28
  • what order filter are you typing about, @Jazzmaniac? 2nd-order biquad, 5 coefficients, 5 degrees of freedom. can't do much more with it. – robert bristow-johnson Nov 20 '14 at 03:01
  • robert bristow-johnson, first of all, thanks for the cookbook. To my knowledge, Orfanidis design is glitchy - it has some weird discontinuities at some frequencies. The design I've purchased is much better than that, so it's doable. I would try to ask for more filter designs from the same designer, but they left the field of DSP. – aleksv Nov 20 '14 at 06:42
  • Jazzmaniac, I'm not much familiar with ODE solvers, but I've used ordinary least squares linear equations solver (via LU decomposition). But I do not think it can solve complex-numbered linear equations, or am I mistaken on this? – aleksv Nov 20 '14 at 06:52
  • Also, the constraint to the equation solver is that resulting X'es have to be real, not complex numbers, so this complicates things a lot. – aleksv Nov 20 '14 at 07:11
  • 1
    @aleksv: You could post a concrete design problem with all specs and see what we come up with. – Matt L. Nov 20 '14 at 08:06
  • @robertbristow-johnson, I'm not talking about biquads or any order of filters here. ODE solvers do not generally map to recursive linear difference equations. Take a 2nd order linear ODE as an analog prototype for example and run a trapezoidal fixed time step solver on it. The result is not a recursive filter but something involving a division for every time step. It cannot be described by a fixed order recursive filter of finite complexity in general. – Jazzmaniac Nov 20 '14 at 10:37
  • @matt-l, sorry I do not know how to formulate it better than that. – aleksv Nov 20 '14 at 12:41
  • @aleksv: OK, so you basically want the same filter types and input parameters as for the Cookbook, just with fewer artefacts at high frequencies, right? But if you used some optimization method, how would you do this in real-time? Or would you pre-compute and store some large set of coefficients for all possible specs? – Matt L. Nov 20 '14 at 13:27
  • Precomputation of coefficients is acceptable. – aleksv Nov 20 '14 at 14:12
  • so alek, what exactly did you purchase? you seem to be talking about a parametric EQ filter and Jazz (and you) seem to be discussing solving differential equations and even non-linear solutions. that's a curiousity. but if what you purchased lead's to a linear combination of $N$ states (recursively or not), it's a filter. then the issue is what are the coefficients in that linear combination and there is no magic that will get you beyond the fact that the number of degrees of freedom are equal to the number of coefficients. not even Andrew Simper can beat that fact. – robert bristow-johnson Nov 20 '14 at 14:40
  • @jazz, we've talked about this at the music dsp site. if what you are modeling with your trapezoidal integration is non-linear, then you have a non-linear system. if what you're modeling is an linear analog prototype, then your trapeziodal integration gets you to no different place than bilinear transform. i proved that easily at the music dsp site. – robert bristow-johnson Nov 20 '14 at 14:43
  • @robert bristow-johnson, I've bought a formula code to calculate biquad peaking/bell filter. It somehow manages to preserve filter's bandwidth at any frequency. There's some inherent reduction of degrees of freedom in Cookbook code like a1=b1 which is absent in the code I've bought (licensed). – aleksv Nov 20 '14 at 14:58
  • @robertbristow-johnson, I don't know what you proved, but at best you did it for a first order trapezoidal integrator, which is trivial. It's higher order ODE solvers that make the game interesting, and they don't map linear ODEs by bilinear transform or in any other way to linear recursive digital filters. – Jazzmaniac Nov 20 '14 at 15:09
  • alek, is what you have left are 5 coefficients? it doesn't have to be the the basic Direct Form DF1 or DF2, it can be built from lattice or state-variable or whatever, but if the transfer function boils down to a 2nd-order rational function, then there are no more 5 coefficients and no more than 5 degrees of freedom. now Knud Christensen has generalized the parametric to 5 degrees of freedom (he tosses in a "tilt" or "symmetry" parameter which rounds it out to 5). but it still does not beat the problem of pushing the peak up against Nyquist. it cannot. – robert bristow-johnson Nov 20 '14 at 15:35
  • robert bristow-johnson, it's doable nevertheless, in various ways (I have 2 different filter calc functions that do that, but they are licensed and I can't disclose them). Of course, these functions only assume a certain gain at Nyquist, they do not optimize the magnitude response derivative at Nyquist, so the form is not ideal, but it is sufficiently good and filter's bandwidth is almost constant across the spectrum, for any filter gain. I just need that for other types of filters. I can further "idealize" the bandwidth by introducing pre-correction formulas. – aleksv Nov 20 '14 at 16:28
  • alek, your language is unpersuasive. the math says that for a 2nd-order IIR filter, there are 5 independent coefficients. no more, nor less (if you want complete generality for a 2nd-order IIR). you have 5 degrees of freedom. with the basic parametric EQ (like in the Cookbook), there are 3 independent parameters, $f_0$, $G_{dB}$, and something related to $Q$ or bandwidth. and the gain at DC and Nyquist are 0 dB. then add an overall gain coefficient that boosts or cuts all frequencies by some $G_0$, that's 4 degrees of freedom. then add the "tilt" or "symmetry" parameter and it's 5. – robert bristow-johnson Nov 20 '14 at 17:17
  • Of course, biquad has 5 degrees of freedom. But this is unrelated to my initial question. – aleksv Nov 20 '14 at 17:40
  • @aleksv, your last statement only demonstrates lack of understanding. is or is not your parametric EQ a 2nd-order IIR or not? – robert bristow-johnson Nov 20 '14 at 18:17
  • lack of understanding? of course it is 2nd order IIR filter I'm talking about. – aleksv Nov 20 '14 at 18:19
  • then, what i have said throughout this entire question or thread stands. it's simply what the math says. if it's 2nd-order (so what Andrew, errr, Jazz says: "In order to effectively compensate the aliasing and warping artefacts for discrete filters you have to use very high orders" does not apply, because you're limited to 2nd-order), then the whole thing is about figuring out what the 5 coefficients are. – robert bristow-johnson Nov 20 '14 at 18:33
  • Yes, of course, the original question IS about finding the 5 coefficients (for each filter type). – aleksv Nov 20 '14 at 18:40
  • okay, Alek. solved problem. been so for 11 years. i hope you're paying your licensing fees to tc electronic, because they own it. – robert bristow-johnson Nov 20 '14 at 18:49
  • What are you talking about? So, Orfanidis also has to pay to tc electronic? Or is his solution somehow different? I can't quite understand what kind of "solution" you are talking about. Formulas can't be licensed, but code can. I licensed code and it has nothing to do with tc electronic in much the same way Orfanidis code is original. – aleksv Nov 20 '14 at 18:57
  • no, Orfanidis precedes Christensen by another about decade. and his solution is not as general. in fact, the Orfanidis solution is precisely a special case of the Christensen generalized solution, at least if you tweek the $Q$ just right (i think Orfanidis may have used my compensation for the bilinear transform scrunching bandwidth, that might change the meaning of $Q$ just a little). – robert bristow-johnson Nov 20 '14 at 19:43
  • @aleksv Hi there, you mentioned you bought a formula code, I wonder would you kindly let me know where did you buy it as I really want to get one to solve my problem. Thank you in advance! – jiandingzhe Jan 22 '18 at 08:52
  • @jiandingzhe Sorry, the guy I've bought it from is no longer associated with audio DSP and probably won't sell the code. – aleksv Feb 04 '18 at 07:47

4 Answers4

6

here is a quick look at how the 5 degrees of freedom for the parametric EQ can be viewed. it's my take on what Knud Christensen of tc electronic came up with about a decade ago at an AES convention and this patent.

so, forget about the Cookbook (and the issues of Q and bandwidth therein) and consider (in the s-plane) the parametric EQ as the sum of a bandpass filter (with a $Q$ value) in parallel with a wire:

$$ H(s) = (G_\text{boost} - 1)\frac{\frac{1}{Q}\frac{s}{\omega_0}}{\left(\frac{s}{\omega_0}\right)^2 + \frac{1}{Q}\frac{s}{\omega_0} + 1} + 1 $$

$G_\text{boost} = 10^{\frac{dB}{20}}$ is the gain of the peak (or valley, if $dB<0$). the gain at DC and at Nyquist is 0 dB. that's a 2nd-order IIR and there are 3 independent parameters. 2 more to go. so we next add an overall gain parameter:

$$ H(s) = G_0 \left((G_\text{boost} - 1)\frac{\frac{1}{Q}\frac{s}{\omega_0}}{\left(\frac{s}{\omega_0}\right)^2 + \frac{1}{Q}\frac{s}{\omega_0} + 1} + 1 \right) $$

that's 4 knobs to twist. one more knob to add (without raising the filter order) and we will be done with adding more independent parameters.

so what Knud does here is replace that "wire" (that trailing "$1$" in the transfer function) with a prototype shelving filter that must have the same poles, the same $Q$ and $\omega_0$ as the BPF, so that the denominator is the same. the transfer function of that shelf is:

$$ H_\text{shelf}(s) = \frac{R\left(\frac{s}{\omega_0}\right)^2 + \frac{\sqrt{R}}{Q}\frac{s}{\omega_0} + 1}{\left(\frac{s}{\omega_0}\right)^2 + \frac{1}{Q}\frac{s}{\omega_0} + 1} $$

where $R \triangleq 10^{\frac{tilt}{20}}$ and $tilt$ is the gain differential of the shelf in dB. this is what offsets the gain at Nyquist to be different than the gain at DC. After bilinear transformation, Nyquist gets boosted by $tilt$ dB and the gain at DC remains unchanged. Like the $dB$ boost parameter, the $tilt$ parameter can be either positive or negative. $G_0 \cdot R$ is the linear gain at Nyquist.

put that all together and you get:

$$ \begin{align} H(s) & = G_0 \left((G_\text{boost} - 1)\frac{\frac{1}{Q}\frac{s}{\omega_0}}{\left(\frac{s}{\omega_0}\right)^2 + \frac{1}{Q}\frac{s}{\omega_0} + 1} + H_\text{shelf}(s) \right) \\ & = G_0 \left((G_\text{boost} - 1)\frac{\frac{1}{Q}\frac{s}{\omega_0}}{\left(\frac{s}{\omega_0}\right)^2 + \frac{1}{Q}\frac{s}{\omega_0} + 1} + \frac{R\left(\frac{s}{\omega_0}\right)^2 + \frac{\sqrt{R}}{Q}\frac{s}{\omega_0} + 1}{\left(\frac{s}{\omega_0}\right)^2 + \frac{1}{Q}\frac{s}{\omega_0} + 1} \right) \\ & = G_0 \frac{R\left(\frac{s}{\omega_0}\right)^2 + (G_\text{boost}+\sqrt{R}-1)\frac{1}{Q}\frac{s}{\omega_0} + 1}{\left(\frac{s}{\omega_0}\right)^2 + \frac{1}{Q}\frac{s}{\omega_0} + 1} \\ \end{align}$$

no matter how you look at that, this has 5 degrees of freedom and those 5 biquad coefficients are fully defined from these 5 parameters. doesn't matter if you map from $s$ to $z$ using the blinear transform or the trapezoidal rule (effectively the same thing) or any other method that does not change the order of the filter. you may have to fudge the definition of $Q$ or bandwidth, you may have to compensate $\omega_0$ and/or $Q$ for frequency warping effects (like you get with the blinear transform), but if you paid big bucks for something that gets you a 2nd-order IIR filter, it doesn't matter if you implement it with any Direct Form or transposed Direct Form or Lattice or Normalized Ladder or Hal Chamberlin's State-Variable or Andrew Simpson's modeling of linear analog with trapezoidal integration, eventually you get to 5 coefficients and they can be mapped to these 5 independent parameters. it's all the same. whether or not you paid money for a license or not. the math is stronger than any claims made by whomever you are licensing from.

Just FYI, i solved where the true peak or valley frequency is when there is a $tilt$ that is not zero. the frequency where the peak or valley has been nudged over by the tilt is:

$$ \omega_\text{peak} \ = \ \omega_0 \ \sqrt{ \frac{Q^2 \left(R - \frac{1}{R}\right)}{G_\text{boost}^2 - R + 2 Q^2 (R - 1)} + \\ \sqrt{\frac{G_\text{boost}^2 - \frac{1}{R} + 2 Q^2 \left(\frac{1}{R} - 1\right)}{G_\text{boost}^2 - R + 2 Q^2 (R - 1)} + \frac{Q^4 \left(R - \frac{1}{R}\right)^2}{\left(G_\text{boost}^2 - R + 2 Q^2 (R - 1)\right)^2} } } $$

you can see that when $tilt = 0$, then $R=1$ and consequently $\omega_\text{peak} = \omega_0$. the peak gain $G_\text{boost}$ might also have to be adjusted a little and that has yet to be worked out. a good first guess would be $G_\text{boost} \leftarrow \frac{G_\text{boost}}{\sqrt{R}}$ or maybe $G_\text{boost} \leftarrow G_\text{boost}-(\sqrt{R}-1)$.

robert bristow-johnson
  • 20,661
  • 4
  • 38
  • 76
  • I cannot quite understand what you are trying to prove? 2nd order IIR filter has 5 degrees of freedom by definition. How your mapping of parameters to biquad coefficients helps to answer the original question? – aleksv Nov 20 '14 at 18:24
  • so, once it became clear that "of course it is 2nd order IIR filter I'm talking about", so that increasing the filter order "to effectively compensate the aliasing and warping artefacts for discrete filters [using] very high orders" is not in the cards, then, as best as i can tell from what you said, is that you paid money to license IP from someone to define those 5 coefficients. and all i am trying to say is that this is a *fully* solved problem. whether you paid for it or not. – robert bristow-johnson Nov 20 '14 at 18:45
  • The peaking filter calc code I have also uses 5 parameters: gain at DC, left edge freq and gain, peak freq and gain. I think it's important to specify edge freq to maintain a stable bandwidth. – aleksv Nov 20 '14 at 18:46
  • It may be a solved problem, but where I can find a solution? Be it paid or not? – aleksv Nov 20 '14 at 18:47
  • okay, now take a look at the Orfanidis paper, but use the math symbols above. this is as good as you can do, because you only have those 5 knobs to twist. you need to figure out what $G_0$ and $A$ are. $\frac{G_0}{A}$ is the DC gain (maybe you want that to be 0 dB or $\frac{G_0}{A} = 1$, i dunno). $A \ G_0$ is the gain at $\omega=\infty$, but, using bilinear transform, that's the gain at Nyquist. so you have to figure out what you want the gain at Nyquist to be. Sophocles has a suggestion. dunno if i agree with it. – robert bristow-johnson Nov 20 '14 at 19:25
  • I have peaking filter calc code already. Orfanidis code is flawed (if doubt try it). – aleksv Nov 20 '14 at 19:39
  • just to let you guys know, i cannot respond to this for another day or two. bye, for now. – robert bristow-johnson Nov 20 '14 at 19:39
  • you will have trouble showing that the Orfanidis solution (the "code" was written by someone else, i am sure) is "flawed" for what it was presented as. i have used it years ago and had no problem if i didn't push it too close to the edge. there is an issue of whether Orfanidis' restriction or setting of gain at the Nyquist frequency is the best idea, but if the resonant frequency gets pushed up to Nyquist, no 2nd-order filter will behave how you want it to behave (like the analog EQ filter). that is because the frequency response must be symmetrical about the Nyquist frequency. – robert bristow-johnson Nov 20 '14 at 19:47
  • Orfanidis code is unusable for general audio equalization as it has "glitches" at certain frequencies where bandwidth is erratically expanded. Maybe in theory its fine. – aleksv Nov 20 '14 at 20:07
  • By the way, the MATLAB code is presented in Orfanidis paper, check it out. It IS glitchy. – aleksv Nov 20 '14 at 20:22
  • upon review of my code, i did find a glitch workaround if the $G_\text{boost} = 1$. a problem of division by zero in the other math for the Orfanidis method. so you have to make sure that $G_\text{boost} < 0.99999$ or $G_\text{boost} > 1.00001$. otherwise, i see no issue. in any case, because of the necessary reflection of the frequency response curve about Nyquist, you will have problems with any discrete-time filter. normally we upsample it (say $f_\text{s}=96\text{ kHz}$) so that we need not worry about Nyquist. – robert bristow-johnson Nov 21 '14 at 14:31
  • I do that G workaround as well, but the glitch is not about it. To really be able to see a glitch you have to be able to move the filter around the frequency and gain range and see the real filter's magnitude response - at some points you'll see the glitch. – aleksv Nov 21 '14 at 16:27
  • while i can modify my answer, i cannot modify my comments. the "$A$" above should be replaced with "$\sqrt{R}$". – robert bristow-johnson Oct 13 '16 at 16:10
3

Using optimization methods, we can get a digital filter's frequency response closer to the target analog filter.

In the following experiment, a 6-order bandpass filter is optimized using Adam, an optimization algorithm often used in machine learning. Frequencies above the passband are excluded from the cost function (assigned zero weight). The optimized filter's response becomes higher than target for frequencies very close to Nyquist, but that difference may be offset by the anti-aliasing filter of the signal source (ADC or sample rate converter). enter image description here enter image description here

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as clr
from scipy import signal

import tensorflow as tf

# Number of sections
M = 3

# Sample rate
f_s = 24000

# Passband center frequency
f0 = 9000

# Number of frequencies to compute
N = 2048

section_colors = np.zeros([M, 3])
for k in range(M):
    section_colors[k] = clr.hsv_to_rgb([(k / (M - 1.0)) / 3.0, 0.5, 0.75])

# Get one of BP poles that maps to LP prototype pole.
def lp_to_bp(s, rbw, w0):
    return w0 * (s * rbw / 2 + 1j * np.sqrt(1.0 - np.power(s * rbw / 2, 2)))

# Frequency response

def freq_response(z, b, a):
    p = b[0]
    q = a[0]
    for k in range(1, len(b)):
        p += b[k] * np.power(z, -k)
    for k in range(1, len(a)):
        q += a[k] * np.power(z, -k)
    return p / q

# Absolute value in decibel

def abs_db(h):
    return 20 * np.log10(np.abs(h))

# Poles of analog low-pass prototype

none, S, none = signal.buttap(M)

# Band limits
c = np.power(2, 1 / 12.0)
f_l = f0 / c
f_u = f0 * c

# Analog frequencies in radians
w0 = 2 * np.pi * f0
w_l = 2 * np.pi * f_l
w_u = 2 * np.pi * f_u

# Relative bandwidth
rbw = (w_u - w_l) / w0

jw0 = 2j * np.pi * f0
z0 = np.exp(jw0 / f_s)

# 1. Analog filter parameters

bc, ac = signal.butter(M, [w_l, w_u], btype='bandpass', analog=True)
ww, H_a = signal.freqs(bc, ac, worN=N)
magnH_a = np.abs(H_a)
f = ww / (2 * np.pi)

omega_d = ww / f_s
z = np.exp(1j * ww / f_s)

# 2. Initial filter design

a = np.zeros([M, 3], dtype=np.double)
b = np.zeros([M, 3], dtype=np.double)
hd = np.zeros([M, N], dtype=np.complex)

# Pre-warp the frequencies

w_l_pw = 2 * f_s * np.tan(np.pi * f_l / f_s)
w_u_pw = 2 * f_s * np.tan(np.pi * f_u / f_s)
w_0_pw = np.sqrt(w_l_pw * w_u_pw)

rbw_pw = (w_u_pw - w_l_pw) / w_0_pw

poles_pw = lp_to_bp(S, rbw_pw, w_0_pw)

# Bilinear transform

T = 1.0 / f_s
poles_d = (1.0 + poles_pw * T / 2) / (1.0 - poles_pw * T / 2)

for k in range(M):
    p = poles_d[k]
    b[k], a[k] = signal.zpk2tf([-1, 1], [p, np.conj(p)], 1)

    g0 = freq_response(z0, b[k], a[k])
    g0 = np.abs(g0)
    b[k] /= g0
    none, hd[k] = signal.freqz(b[k], a[k], worN=omega_d)

plt.figure(2)
plt.title("Initial digital filter (bilinear)")

plt.axis([0, f_s / 2, -90, 10])

plt.plot(f, abs_db(H_a), label='Target response', color='gray', linewidth=0.5)

for k in range(M):
    label = "Section %d" % k
    plt.plot(f, abs_db(hd[k]), color=section_colors[k], alpha=0.5, label=label)

# Combined frequency response of initial digital filter

Hd = np.prod(hd, axis=0)
plt.plot(f, abs_db(Hd), 'k', label='Cascaded filter')
plt.legend(loc='upper left')

plt.figure(3)
plt.title("Initial filter - poles and zeros")
plt.axis([-3, 3, -2.25, 2.25])
unitcircle = plt.Circle((0, 0), 1, color='lightgray', fill=False)
ax = plt.gca()
ax.add_artist(unitcircle)

for k in range(M):
    zeros, poles, gain = signal.tf2zpk(b[k], a[k])
    plt.plot(np.real(poles), np.imag(poles), 'x', color=section_colors[k])
    plt.plot(np.real(zeros), np.imag(zeros), 'o', color='none', markeredgecolor=section_colors[k], alpha=0.5)

# Optimizing filter

tH_a = tf.constant(magnH_a, dtype=tf.float32)

# Assign weights

weight = np.zeros(N)
for i in range(N):
    # In the passband or below?
    if (f[i] <= f_u):
        weight[i] = 1.0

tWeight = tf.constant(weight, dtype=tf.float32)
tZ = tf.placeholder(tf.complex64, [1, N])

# Variables to be changed by optimizer
ta = tf.Variable(a)
tb = tf.Variable(b)
ai = a
bi = b

# TF requires matching types for multiplication;
# cast real coefficients to complex
cta = tf.cast(ta, tf.complex64)
ctb = tf.cast(tb, tf.complex64)

xb0 = tf.reshape(ctb[:, 0], [M, 1])
xb1 = tf.reshape(ctb[:, 1], [M, 1])
xb2 = tf.reshape(ctb[:, 2], [M, 1])

xa0 = tf.reshape(cta[:, 0], [M, 1])
xa1 = tf.reshape(cta[:, 1], [M, 1])
xa2 = tf.reshape(cta[:, 2], [M, 1])

# Numerator:   B = b₀z² + b₁z + b₂
tB = tf.matmul(xb0, tf.square(tZ)) + tf.matmul(xb1, tZ) + xb2

# Denominator: A = a₀z² + a₁z + a₂
tA = tf.matmul(xa0, tf.square(tZ)) + tf.matmul(xa1, tZ) + xa2

# Get combined frequency response
tH = tf.reduce_prod(tB / tA, axis=0)

iterations = 2000
learning_rate = 0.0005

# Cost function
cost = tf.reduce_mean(tWeight * tf.squared_difference(tf.abs(tH), tH_a))
optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)

zz = np.reshape(z, [1, N])

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    for epoch in range(iterations):
        loss, j = sess.run([optimizer, cost], feed_dict={tZ: zz})
        if (epoch % 100 == 0):
            print("  Cost: ", j)

    b, a = sess.run([tb, ta])

for k in range(M):
    none, hd[k] = signal.freqz(b[k], a[k], worN=omega_d)

plt.figure(4)
plt.title("Optimized digital filter")

plt.axis([0, f_s / 2, -90, 10])

# Draw the band limits
plt.axvline(f_l, color='black', linewidth=0.5, linestyle='--')
plt.axvline(f_u, color='black', linewidth=0.5, linestyle='--')

plt.plot(f, abs_db(H_a), label='Target response', color='gray', linewidth=0.5)

Hd = np.prod(hd, axis=0)
for k in range(M):
    label = "Section %d" % k
    plt.plot(f, abs_db(hd[k]), color=section_colors[k], alpha=0.5, label=label)

magnH_d = np.abs(Hd)
plt.plot(f, abs_db(Hd), 'k', label='Cascaded filter')
plt.legend(loc='upper left')

plt.figure(5)
plt.title("Optimized digital filter - Poles and Zeros")
plt.axis([-3, 3, -2.25, 2.25])
unitcircle = plt.Circle((0, 0), 1, color='lightgray', fill=False)
ax = plt.gca()
ax.add_artist(unitcircle)

for k in range(M):
    zeros, poles, gain = signal.tf2zpk(b[k], a[k])
    plt.plot(np.real(poles), np.imag(poles), 'x', color=section_colors[k])
    plt.plot(np.real(zeros), np.imag(zeros), 'o', color='none', markeredgecolor=section_colors[k], alpha=0.5)

plt.show()
igorinov
  • 366
  • 1
  • 4
  • "Frequencies above the passband are excluded from the cost function (assigned zero weight)" Isn't there a danger that the filter becomes a sort of high-pass filter? Frequencies above passband could have large gain with no penalty given by such a cost function. – Olli Niemitalo Jan 21 '19 at 08:48
  • OK, II thought a symbol regression method would produce a finalized equation...probably for most cases,not all. while real-time derivative-free? optimization is a bit slow – aleksv Jan 22 '19 at 04:15
  • 1
    @OlliNiemitalo: For some frequencies, it drops to zero after passband, like the initial filter. – igorinov Jan 22 '19 at 06:36
  • 1
    @aleksv: There is another closed form solution that has not been mentioned here - http://www.khabdha.org/wp-content/uploads/2008/03/optimizing-the-magnitude-response-of-mzt-filters-mzti-2007.pdf – igorinov Jan 22 '19 at 06:42
  • 1
    With tensorflow-gpu 1.12.0 the optimization is not converging, rather diverging. tensorflow 1.12.0 works better but the optimization does not fully converge, because of the choice of the optimizer (Adam) and its learning rate, and possibly because of the way the cost function is formulated. I will award the bounty to this answer, because it shows a modern approach to optimization of coefficients: automatic differentiation. I would still like to see: more stable optimization and arranging obtained coefficients for real-time use (such as parameter sweeps). – Olli Niemitalo Jan 22 '19 at 08:22
  • Hi igorinov, I'm confused with your code that it seems tZ is just a placeholder that initialized to zero, and you use it directly and B = b₀z² + b₁z + b₂, A = a₀z² + a₁z + a₂ to calculate the frequency response. Am I missing something? – ZR Han Feb 08 '22 at 09:48
  • @ZRHan At that point, those operations are added to the graph without calculating anything yet. https://www.tensorflow.org/guide/intro_to_graphs I should port that example to TensorFlow 2 and Keras... – igorinov Feb 10 '22 at 05:11
  • @igorinov OK, thanks! I got it. – ZR Han Feb 10 '22 at 05:44
2

@Jazz, one of the things we learned in electrical engineering is that any order of differential equation can be broken down to a set (or "system") of 1st-order diff eqs. so if trapezoidal integration, with the same "time step" $\Delta t$ is being used consistently for all continuous-time integrals, for a $N$th-order linear ODE, you can bust that up into $N$ first-order differential equations. then consider just one of those 1st-order diff eqs:

again, consider emulating a capacitor. let the sampling period be $T=\frac{1}{f_\text{s}}$ be the same as the "$\Delta t$" used in the trapezoid rule.

$$ i(t) = C \frac{dv}{dt} $$

or

$$ v(t) = \frac{1}{C} \int\limits_{-\infty}^{t} i(u) \ du $$

in the s-domain it's

$$ V(s) = \frac{1}{s} \left( \frac{1}{C} I(s) \right) $$

so trapazoidal integration at discrete times is:

$$\begin{align} v(nT) & = \frac{1}{C} \int\limits_{-\infty}^{nT} i(u) \ du \\ & = \frac{1}{C} \sum\limits_{k=-\infty}^{n} \quad \int\limits_{kT-T}^{kT} i(u) \ du \\ & \approx \frac{1}{C} \sum\limits_{k=-\infty}^{n} \frac{T}{2} \left( i(kT-T) + i(kT) \right) \\ & = v((n-1)T) + \frac{1}{C} \frac{T}{2} \left( i((n-1)T) + i(nT) \right) \\ \end{align}$$

or as discrete-time sample values

$$ v[n] = v[n-1] + \frac{T}{2C} (i[n] + i[n-1]) $$

applying the Z transform

$$ V(z) = z^{-1} V(z) \ + \ \frac{T}{2C} \left(I(z) + z^{-1} I(z) \right) $$

solving for $V$

$$ V(z) = \frac{T}{2} \frac{1 + z^{-1}}{1 - z^{-1}} \left( \frac{1}{C} I(z) \right) $$

looks like we're substituting

$$ \frac{1}{s} \leftarrow \frac{T}{2} \frac{1 + z^{-1}}{1 - z^{-1}} $$

or

$$ s \leftarrow \frac{2}{T} \frac{z-1}{z+1} $$

which is precisely what the bilinear without compensation for frequency warping does.

robert bristow-johnson
  • 20,661
  • 4
  • 38
  • 76
  • This is what is usually called "proof by example" and hardly general. It also only treats the case that I already said was trivial in the comments above. Again, I'm talking about higher order implicit solvers that don't give a simple discrete recursion. It's also hardly the right place for this discussion as your answer has nothing to do with the original question. If you move this to a new question regarding ODE integration I'm willing to contribute to it. – Jazzmaniac Nov 20 '14 at 18:13
  • 1
    no, it's completely general for linear diff eqs. you are ignoring the opening statement about we learned long ago in electrical engineering classes. (both in numerical methods and in control theory, an $N$th-order differential equation can be broken down to $N$ first-order differential equations. and then i require consistency; that the same trapezoidal rule for integration is used throughout.) – robert bristow-johnson Nov 20 '14 at 18:29
  • I am fully aware of the transformation of higher order ODEs into systems of first order ODEs. And again, it is not what I'm talking about. This statement doesn't make your proof any more general, as you just assert that the bilinear mapping transcends to the entire class while only treating a simple single example. Still, I'm not saying that your result is wrong, I'm talking about different solvers. Higher order solvers are not solvers of higher order DEs. – Jazzmaniac Nov 20 '14 at 18:46
  • sorry, Andrew. that's a fail. you truly do not understand if you're continuing to maintain that modeling the continuous-time behavior of the linear analog components using trapezoidal integration, with the step time $\Delta t$ being the same as the sampling period $T=\frac{1}{f_\text{s}}$, then that is the very same thing as using bilinear transform without compensation for frequency warping. same thing and the proof is above. – robert bristow-johnson Nov 20 '14 at 19:18
  • Sorry Robert, I'm not Andrew. And your proof above is like the following. I have two maps A and B, and some x. Both A(x)=c and B(x)=c, therefore A = B. That's obviously wrong. Even if A and B are linear, it's still wrong. You're right that a first order trapezoidal solver on a linear system will generate a recursive discrete system, but that's where it ends. You keep ignoring my repeated statements about higher order (adaptive/implicit) integrators. So I see the fail very much on your side. – Jazzmaniac Nov 20 '14 at 19:24
  • okay, Jazz, start with a linear ODE of order $N$ and break it up, using the standard method, into $N$ first simple first-order linear differential equations. then Laplace transform all of those 1st-order diff eqs. now, in the s-domain, you will only see powers of $s$ as in $s^{-1}$. but you know that when you recombine the $N$ 1st-order ODEs into the original $N$th-order ODE you will get your higher powers back. but each of those $s^{-1}$ factors will be converted to $\frac{T}{2}\frac{z+1}{z-1}$ because that's what trapezoidal integration does. and what bilinear does. – robert bristow-johnson Nov 20 '14 at 19:37
  • Then why don't you add all those details to your answer and make it a proof that deserves the name? – Jazzmaniac Nov 20 '14 at 20:03
  • naw, it's good enough the way it is. what is clear, when you break up your $N$th-order diff. eq. into $N$ first-order diff eqs., then every occurrence of $s^{-1}$ is replaced by $\frac{T}{2}\frac{z+1}{z-1}$, and remain so as you recombine these to higher powers of $s$. that is precisely the same as what bilinear transform without compensation for frequency warping does. – robert bristow-johnson Nov 21 '14 at 14:43
1

I've come up with a design for the 10dB peaking EQ. I've chosen 20 filters with center frequencies between 500 Hz and 16 kHz (Fs = 48 kHz). The top plot below is the design according to RBJ's Audio-EQ-Cookbook, which is good but which leads to bandwidth distortion when the center frequencies get closer to Nyquist. The bottom plot is the new design where the filters very closely match the analog prototype filters: enter image description here

And this is how the new notch filters look like compared to the Cookbook (bandwidth = 4 octaves, highest $f_0=23$ kHz): enter image description here

The following figure shows a lowpass filter design ($Q=2$, $f_0=16$ kHz, $F_s=48$ kHz). Note that the new design approximates the analog prototype and for this reason it does not perform as a conventional lowpass filter (it has no zero at Nyquist):

enter image description here

robert bristow-johnson
  • 20,661
  • 4
  • 38
  • 76
Matt L.
  • 89,963
  • 9
  • 79
  • 179
  • Matt, i might suggest plotting the "peakingEQ" for, say, a 10 dB boost with a fixed $BW$, say 1 octave, and for a variety of peak frequencies, going up to close to Nyquist. even though the Cookbook does a first-order compensation on $BW$ by multiplying by $\frac{\omega_0 T}{\sin(\omega_0 T)}$, it still cannot change the fact that the frequency response must be continuous and have continuous derivatives and it must be symmetrical about Nyquist. even if you bump up the gain at Nyquist as does Orfanidis. that, in and of itself, is the source of this warped frequency response. – robert bristow-johnson Nov 21 '14 at 14:20
  • 1
    i think he's expecting to see a digital filter, with a sample rate as low as 44.1 kHz, match the analog filter, even if the resonant frequency is just below Nyquist. this is something i had been worrying about 2 decades ago and all i could think of doing easily is simply to compensate for the scrunching of bandwidth that bilinear transform does. then Orfanidis changed one assumption (that the gain at Nyquist is 0 dB), and that helped a little more. but the thing that really helps, is simply to bump up the sample rate. – robert bristow-johnson Nov 21 '14 at 15:28
  • Thanks for an attempt. The first Cookbook plot is what I want, but as Robert also noted, with the gain at Nyquist being non-zero. It will be also beneficial to draw the responses on the logarithmic scale in the range e.g. 20 Hz to Nyquist. Audio EQs rarely use a linear frequency scale. And bandwidth is usually specified in octaves. Any biquads have no problems with center/corner frequency up to about 300 Hz, but going higher reveals the issue. – aleksv Nov 21 '14 at 16:23
  • No, I mean 0 to 300 Hz, lower frequencies. Above 300 Hz up to Nyquist these Cookbook filter formulas produce progressively increasing filter shape distortion. Of course, the sample rate is 44100 or 48000 as usual. – aleksv Nov 22 '14 at 10:57
  • You have to plot the graph on the log scale. It is bandwidth distortion on the LOG scale what bothers me. There's no distortion present in the lower frequencies when seen on the log scale. Peaking, low-pass, high-shelf, notch filters always have 0 dB (unity) gain at DC. – aleksv Nov 22 '14 at 12:36
  • @MattL. , for peakingEQ, it's two parallel filters summed, one a BPF (with gain of $(G_\text{boost}-1)$) and the other a wire (gain of $1$). at DC the gain should be 0 dB. at $\omega=\infty$, if it were an analog peakingEQ filter, the gain should also be 0 dB. but, using the bilinear transform, this $\infty$ frequency gets mapped to Nyquist. so what both Orfanidis and Christensen do (and latter is more general), is they still use bilinear but design the analog gain at $\infty$ to be something other than the gain at DC. – robert bristow-johnson Nov 22 '14 at 17:13
  • @MattL, yes, it looks exactly what's needed (at least for peaking filter). I would "zoom in" into the area 500 Hz to Nyquist and increase the number of magnitude plots, to see more variation. – aleksv Nov 23 '14 at 07:20
  • @aleksv: I've designed some peaking EQ filters that are pretty close to the analog prototype filters. Please check my edited answer. – Matt L. Nov 23 '14 at 15:05
  • @MattL, this is great (only maybe a small peak gain overshoot is visible). Is your design procedure streamlined? I mean, can it be represented in C code? This may be needed to build interpolation tables for use in real-time. – aleksv Nov 23 '14 at 15:23
  • @aleksv: This is just a proof of concept, the gain overshoot can of course be corrected very easily. The design method is probably best suited for offline pre-computation of coefficients. Anyway, I'm not planning on disclosing the method because it is a novel approach as far as I know. If you're interested in sets of coefficients, you can contact me (see my profile). – Matt L. Nov 23 '14 at 21:41
  • @aleksv: I added a plot showing the new notch filters in comparison with the Cookbook design. – Matt L. Nov 25 '14 at 10:56
  • I would also test your algorithm with some higher bandwidth value like 4 octaves. – aleksv Nov 25 '14 at 12:14
  • @aleksv: I changed the notch filter example to a bandwidth of 4 octaves. – Matt L. Nov 25 '14 at 13:58
  • Looks good. What about drawing a little more curves up to 0.98 of Nyquist, not up to just 16kHz? Also as far as I can see, Notch's bandwidth in the lower frequencies is a bit imprecise, but I guess it's toleratable deviation. – aleksv Nov 25 '14 at 15:12
  • @aleksv: Hey, this is fun ... I'm not sure if this is useful, but it can be done. Have a look. I guess by now it should be clear that the method actually works and produces results that are very different from methods based on the bilinear transform. – Matt L. Nov 25 '14 at 15:39
  • Can you also design low-, high-pass, high-shelf and band-pass filters? – aleksv Nov 27 '14 at 15:10
  • @aleksv: Yes, the method works regardless of the filter type. For lowpass and bandpass filters it is a matter of taste what one likes, because approximating the analog filter might not be the best choice in these cases, because an analog lowpass filter with a low Q factor might have hardly any attenuation at Nyquist, so its digital counterpart is actually no lowpass filter at all. Here it's important to define what the desired response actually is, but I have ideas about that. – Matt L. Nov 27 '14 at 15:31
  • Well, since the goal is to have a constant bandwidth, it does not matter if low-pass becomes something else at Nyquist. Cookbook's prototypes are quite good. The Q factor is usually equated to the Gain - this way it becomes easy to manipulate the filter. E.g. set Gain to -3.01dB (0.707 linear) for a "standard" low- or high-pass filter. – aleksv Nov 27 '14 at 16:37
  • @aleksv: OK, if the digital filter is supposed to approximate the analog prototype in all cases, then the design goal is at least very clear. Check out the new figure in my answer above. – Matt L. Nov 27 '14 at 17:35
  • hey Matt, this is interesting. so it's a LPF with a reasonably high Q (to get a bump), and since the bump height is directly related to $Q$ (in the s-domain, the peak is at $\omega_0\sqrt{1-\frac{1}{2Q^2}}$ and the peak height (assuming the DC gain is 0 dB) is $\frac{Q}{\sqrt{1-\frac{1}{4Q^2}}}$) we know that the bilinear transform will not change the peak height, so to get the different peak "sharpness" (looks like the Cookbook got it too sharp), you need to do a different mapping of $Q$. the Cookbook leaves $Q$ unchanged, but it does modify the mapping from $BW$ to $Q$ a little. – robert bristow-johnson Nov 30 '14 at 17:56
  • @robertbristow-johnson: There are two reasons for differences between the Cookbook design and the prototype lowpass filter: one is that the Cookbook gets too high a Q value for large $f_0$ (due to the bilinear transform), and the other one is the constraint that a Cookbook lowpass has both its zeros at Nyquist. – Matt L. Nov 30 '14 at 18:09
  • beg to differ, Matt, at least on semantics. the Cookbook's use of bilinear transform is quite vanilla-flavored use with the standard compensation for the "principal frequency" (in the LPF case, it's the resonant frequency, $\omega_0$) with one more twist, and that is a teeny bit of compensation for scrunching the bandwidth. but $Q$ remains uncompensated. however, the relationship between $Q$ and bandwidth is tweaked a little. so the height of the bump must be the same (as it is with bilinear), but the location (relative to $\omega_0$) and width of the bump is adjusted. – robert bristow-johnson Nov 30 '14 at 18:16
  • and you're right about the two zeroes at Nyquist, which are mapped from $\infty$ to $\pi$ by the bilinear transform. but that Knud Christensen thing does provide a degree of freedom to adjust that thing. so your design should be a special case of the Christensen design (that has 5 degrees of freedom), where $G_\text{shelf}$, $G_0$, and $Q$ (and maybe the other two parameters) are adjusted by some mapping that your secret sauce knows about. – robert bristow-johnson Nov 30 '14 at 18:26
  • @robertbristow-johnson: OK, I know that there's some bandwidth compensation in the Cookbook formulas, but still, you can't help getting squeezed responses if you move towards Nyquist, that's the curse of BLT. And as for Christensen, I only read what you wrote in your answer because I don't have access to that paper, but if there are 5 degrees of freedom then, well, any biquad can be called a special case of that design. – Matt L. Nov 30 '14 at 20:17
  • precisely. it's just that for some biquads (like a LPF with gain at Nyquist going to $-\infty$ dB) some of those parameters will also go to some version of $\infty$. my curiosity is that your biquad filter should be able to be inverse bilinear-transform mapped back to the s-domain and that should be expressible with the bottom equation i have in that answer regarding Christensen's solution. lemme know an email address and i'll send a copy, but i think my synopsis is more succinct. that bottom s-domain equation has 5 independent parameters and should fit any 2nd-order $H(s)$. – robert bristow-johnson Nov 30 '14 at 20:39
  • @robertbristow-johnson: It would be great if you sent a copy! Please find my address via http://mattsdsp.blogspot.nl/ ('About Me'). – Matt L. Nov 30 '14 at 20:46
  • @MattL. , are you publishing your solution somewhere? (or is it something that you sell?) i would like to try to come up with expressions for 5 parameters in what i posted here in terms of your solution. i can see that you're keeping the peak frequency compensated and adjusting $Q$ so that the second derivative is also fixed, after frequency warping. but either way, i see this as a 5 knob problem. – robert bristow-johnson Dec 13 '15 at 19:12
  • @robertbristow-johnson: Haven't sold or published it yet. I don't have an analytic solution, I have an optimization method where I compute a discrete-time approximation of the analog prototype. – Matt L. Dec 13 '15 at 19:47
  • oh, okay. not closed-form. with the $\frac{\omega_0}{\sin(\omega_0)}$ bandwidth adjustment (which is accurate for thin bandwidths) in th cookbook, i was expecting to get the 2nd derivative well matched around the peak. and given a "$tilt$" parameter, i can see how i would compensate the resonant frequency so that the peak remains unchanged, but i am still not certain what to set the "$tilt$" to be. there is the "Orfanidis setting", but it should be set higher than that, and your optimized plots show that. – robert bristow-johnson Dec 13 '15 at 19:55
  • @MattL. did you get a copy of the Christenson paper? i thought i got it to you, but i don't remember. – robert bristow-johnson Dec 13 '15 at 20:53
  • @robertbristow-johnson: Yes, I got it, thanks. – Matt L. Dec 13 '15 at 20:59
  • @MattL. Hi there, your "new design" looks amazing! Would you kindly let me know about your new design? I really want to do it too, thank you in advance! – jiandingzhe Jan 22 '18 at 08:53