Sometimes I see some links to websites, and they are bankrupt or gone.
Then I do an archive.org search, and find the old content, sometimes it's good stuff.
Then I check on copyscape and Google if the content is indexed and it's usually gone.
Is it ok to use this on my website? Can I get penalized for duplicate content?
Not sure this question is good for this, but don't know where else to ask!
It is not legal to put content from defunct websites onto your own website. In US law this is a copyright violation. Everything written automatically gets copyright protection. Only very old content (well pre-dating the internet era) has fallen out of copyright protection.
This is very similar to Abandonware. That is computer software and games that are no longer published. This article sums it up very nicely:
Is Abandonware Legal?
The simple answer is a definitive no. Copyrighted works that have been abandoned by their creators do not automatically become public property. On the contrary, they remain protected for the duration of the copyright, which varies from country to country. Downloading a copyrighted work is illegal, too, so it’s not just the sites that could get in trouble.
