The previous answer was incorrect due to an incorrect use of majorization. I decided to replace it by a correct version below for completeness, and to remedy the embarrassing error (though now the answer becomes essentially equivalent to the OPs own answer, so feel free to ignore!).
$\newcommand{\da}{\downarrow} \newcommand{\ua}{\uparrow} \newcommand{\nlsum}{\sum\nolimits}$
Recall that for Hermitian matrices $A$ and $B$, Lidskii's eigenvalue majorization inequality implies that there exist doubly stochastic matrices $D_1$ and $D_2$ such that $\lambda(A+B)=D_1\lambda(A)+D_2\lambda(B)$.
Introduce the shorthand $a_{m+1}=\lambda(A_1+\cdots+A_m)$, and $a_j = \lambda(A_j)$ for $1\le j\le m$. Then, applying Lidskii's result repeatedly and noting that doubly stochastic matrices are closed under multiplication, it follows that there exist doubly stochastic matrices $D_1,\ldots,D_m$ such that
\begin{equation*}
a_{m+1} = D_1a_1+\cdots+D_ma_m.
\end{equation*}
We wish to upper bound $\det(A_{m+1}):=\prod_i a_{m+1,i}$. Consider the function
\begin{equation*}
f(D_1,\ldots,D_m) := \prod_i (D_1a_1+\cdots+D_ma_m)_i. % \le \left(\frac{\nlsum_i e_i^T(D_1a_1+\cdots+D_ma_m)}{n} \right)^n.
\end{equation*}
We now show that $f$ is maximized over permutation matrices. First, we make the change of variables $(D_l)_{ij}=D_{l,ij} = e^{u_{l,ij}}$. Moreover, we relax the doubly-stochastic constraints on each $D_l$ to
\begin{equation*}
\Omega_n := \left\lbrace(U_1,\ldots,U_m) \in \mathbb{R}^{m\times n \times n}\mid \nlsum_i e^{u_{l,ij}} \le 1,\ \forall i,\quad \nlsum_j e^{u_{l,ij}} \le 1,\forall j, \quad\text{for}\ 1 \le l \le m\right\rbrace.
\end{equation*}
Then, we consider the optimization problem
\begin{equation*}
\sup_{(U_1,\ldots,U_m) \in \Omega_n}\quad g(U_1,\ldots,U_m) := \nlsum_{i=1}^n \log\left(\nlsum_{lj} a_{l,j}e^{u_{l,ij}}\right).
\end{equation*}
Since all the matrices $A_1,\ldots,A_m$ are positive definite, each $a_{l,j}\ge0$. Thus, $g$ is a convex function of the $U_i$. Hence its supremum will be at an extreme point of $\Omega_n$. These points correspond to the permutation matrices. Thus, mapping back to the $D_i$ space, we observe that $f$ will be maximized at permutation matrices.
This reasoning immediately yields the desired inequality:
\begin{equation*}
\det(A_{m+1})=\det(A_1+\cdots+A_m) \le \sup_{\sigma_1,\ldots,\sigma_m \in S_n}\prod_{i=1}^n\left(\nlsum_{l=1}^m a_{l,\sigma_l(i)} \right).
\end{equation*}