I am assuming that the following facts do not require proofs for the purposes of this question.
Fact 1: Let $h_n$ be a sequence in $\mathbb{R}^K$ such that $\lim_{n\rightarrow \infty} h_n =h\in \mathbb{R}^K$. Then, for each $i\in \{1,2,\ldots,K\}$, we have $h^i_n\rightarrow h^i$.
Fact 2: Let $z_n$ and $q_n$ be sequence in $\mathbb{R}$ such that $\lim_{n\rightarrow \infty} z_n =z\in \mathbb{R}$ and $\lim_{n\rightarrow \infty} q_n =q\in \mathbb{R}$. Then,
i) $\lim_{n\rightarrow \infty}(z_n\pm q_n)=z\pm q$
ii) $\lim_{n\rightarrow \infty}(z_n \times q_n)=z\times q$
Fact 3: Let $z_n$ in $\mathbb{R}$ such that for each $n$, $z_n\leq a\in \mathbb{R}$ and $\lim_{n\rightarrow \infty} z_n =z$. Then, $z\leq a$.
Definition: A correspondence $G:X\rightrightarrows Y $ is upper hemicontinuous at $x\in X$ if for any open neighborhood of $V$ of $G(x)$. There exists a neighborhood $U$ of $x$ such that for all $x'$ in $U$, $G(x')\subseteq V$.
Unlike the previous remarks, the following fact requires proof.
Fact 4: Let $G:X\rightrightarrows Y $ be a correspondence. If for every $x_n\rightarrow x\in X$ and $y_n\in G(x_n)$ there exists a subsequence $y_{n_k}$ of $y_n$ with $y_{n_k}\rightarrow y$ and $y\in G(x)$, then $G$ is upper hemicontinuous. Moreover, if $G$ is compact valued, then the converse is also true.
Now, let's go back to your question. Let $(p_n,w_n)$ be a sequence in $\mathbb{R}^{L+1}_{++}$ such that $\lim_{n\rightarrow \infty} (p_n,w_n) =(p,w)\in \mathbb{R}^{L+1}_{++}$. Moreover, let $x_n\in B(p_n,w_n)$ for all $n$.
Let $w^\ast = \sup_n w_n$. Since $w_n\rightarrow w\in \mathbb{R}_{++}$, it must be true that $w^\ast \in \mathbb{R}_{++}$ because otherwise we could have created a subsequence $w_{n_k}$ of $w_n$ for which $w_{n_k}\rightarrow\infty$ which would contradict with $w_n\rightarrow w$. Similar arguments would imply that $p^{\ast} = \min_i( \inf_n p^i_n)>0$ since $p^n\rightarrow p\in\mathbb{R}_{++}^L$.
The previous observations imply that for all $n$ and for all $i$ we have $x_n^i\leq w^\ast/p^\ast$. Therefore, the sequence $x_n$ is bounded. By Bolzano-Weierstrass Theorem, $x_n$ has a convergent subsequence $x_{n_k}$ with $x_{n_k}\rightarrow x$. From here onwards, we will suppress the subscript $k$.
Since $x_n\in B(p_n,w_n)$, we have $\sum_i p_n^i\times x_n^i-w_n\leq 0$. Let $c_n = \sum_i p_n^i\times x_n^i$. By fact 1, we have $p_n^i\rightarrow p_i$ and $x_n^i\rightarrow x_i$ for each $i$, and by fact 2, we have $c_n=\sum_i p_n^i\times x_n^i\rightarrow p\cdot x$.
By fact 2, $c_n-w_n\rightarrow (p\cdot x-w)$. Fact 3 implies that $(p\cdot x-w)\leq 0$, which in turn implies that $x\in B(p,w)$. This concludes the proof that the budget correspondence is upper hemicontinuous.
Original (incorrect) Answer
The following answer has been posted earlier. However, I realized that there is a mistake with this one. I am keeping this here since the original question had been wondering specifically about how Bolzano-Weierstrass theorem is relevant to the question and comparison of this incorrect answer and the correct version shows why we would need this theorem.
Definition: A correspondence $G:X\rightrightarrows Y $ is upper hemicontinuous at $x\in X$ if for all sequences $x_n\in X$ with $x_n\rightarrow x$, for all $y_n\in G(x_n)$ with $y_n \rightarrow y \in Y$, we have $y\in G(x)$.
Now, let's go back to your question. Let $(p_n,w_n)$ be a sequence in $\mathbb{R}^{L+1}$ such that $\lim_{n\rightarrow \infty} (p_n,w_n) =(p,w)$. Moreover, let $x_n\in B(p_n,w_n)$ for all $n$, with $x_n\rightarrow x$.
Since $x_n\in B(p_n,w_n)$, we have $\sum_i p_n^i\times x_n^i-w_n\leq 0$. Let $c_n = \sum_i p_n^i\times x_n^i$. By fact 1, we have $p_n^i\rightarrow p_i$ and $x_n^i\rightarrow x_i$ for each $i$, and by fact 2, we have $c_n=\sum_i p_n^i\times x_n^i\rightarrow p\cdot x$.
By fact 2, $c_n-w_n\rightarrow (p\cdot x-w)$. Fact 3 implies that $(p\cdot x-w)\leq 0$, which in turn implies that $x\in B(p,w)$. This concludes the proof that the budget correspondence is upper hemicontinuous.
