In order to use the subfigure environment you need subcaption. For \includegraphics you need graphicx. For the ngerman option you need babel.
Next, \vspace*{\fill} and \vfill inside subfigure do nothing at all. Also, subfigure doesn't know what to do with the [h] option: it accepts [t], [b] or [c] (the last one being the default) to specify the vertical appearance of the built box; in this case you need [c] (hence you can omit it).
Using scale=0.5 is wrong: you most likely want to scale the image to the size given for subfigure. With scale you have no idea about the final size of the image, so width=\linewidth is preferable, because TeX will use the size reserved for subfigure.
You also need to fix the labels.
If you need some vertical space between the image, add it. The trick is to have a blank line between the subfigure environments (with a \vspace instruction in between).
\documentclass[
ngerman,
12pt,
a4paper,
oneside,
listof=totoc,
plainfootsepline,
]{scrbook}
\usepackage{scrlayer-scrpage}
\usepackage{babel}
\usepackage{graphicx}
\usepackage{subcaption}
\addtokomafont{disposition}{\rmfamily}
\addtokomafont{captionlabel}{\bfseries}
\RedeclareSectionCommand[
beforeskip=-.5\baselineskip,
afterskip=.25\baselineskip
]{subsubsection}
\begin{document}
\begin{figure}[htp]
\centering
\begin{subfigure}{0.5\textwidth}
\centering
\includegraphics[width=\linewidth]{example-image}
\caption{fig1}
\label{subfig:fig1}
\end{subfigure}
\vspace{3ex}
\begin{subfigure}{0.5\textwidth}
\centering
\includegraphics[width=\linewidth]{example-image}
\caption{fig2}
\label{subfig:fig2}
\end{subfigure}
\caption{fig2}
\label{fig:fig}
\end{figure}
\end{document}

If you really need scale, you can use \subcaptionbox that doesn't need a width to begin with.
\documentclass[
ngerman,
12pt,
a4paper,
oneside,
listof=totoc,
plainfootsepline,
]{scrbook}
\usepackage{scrlayer-scrpage}
\usepackage{babel}
\usepackage{graphicx}
\usepackage{subcaption}
\addtokomafont{disposition}{\rmfamily}
\addtokomafont{captionlabel}{\bfseries}
\RedeclareSectionCommand[
beforeskip=-.5\baselineskip,
afterskip=.25\baselineskip
]{subsubsection}
\begin{document}
\begin{figure}[htp]
\centering
\subcaptionbox{fig1\label{subfig:fig1}}{%
\includegraphics[scale=0.5]{example-image}%
}
\vspace{3ex}
\subcaptionbox{fig2\label{subfig:fig2}}{%
\includegraphics[scale=0.5]{example-image}%
}
\caption{fig2}
\label{fig:fig}
\end{figure}
\end{document}
The plainfootsepline option only makes sense if you load scrlayer-scrpage.