In modern あいうえお, the column of わ is わいうえを, but it used to be わゐうゑを. The pronunciation of these were more like "we" and "way" as opposed to the normal い and え sound. I didn't know this myself, but apparently these old forms have disappeared as recent as 1946.
ゝ is a short-hand that means "repeating the previous letter", such as 学問のすゝめ (学問のすすめ). As you probably know, dakuten is composable decoration of a letter, and it can also decorate ゝ.
ヶ is an interesting one. I think its origin is at least in part katakana ケ, but I can't think of other katakanas that do this. Aside from the usage in 藤ヶ谷, which means 藤の丘, you also use ヶ to count objects, like 10ヶ.
it used to be わゐうゑをSee https://ja.wikipedia.org/wiki/%E3%82%84%E8%A1%8C%E3%81%84 - う had its own kana. – dROOOze Jun 02 '18 at 01:10