Although the question was asking about the Object.finalize method, the subject really is about the finalization mechanism as a whole. This mechanism includes not only the surface API Object.finalize, but it also includes specifications of the programming language about the life cycle of objects, and practical impact on garbage collector implementations in JVMs.
Much has been written about why finalization is difficult to use from the application's point of view. See the questions Why would you ever implement finalize()? and Should Java 9 Cleaner be preferred to finalization? and their answers. See also Effective Java, 3rd edition by Joshua Bloch, Item 8.
Briefly, some points about the problems associated with using finalizers are:
they are notoriously difficult to program correctly
in particular, they can be run unexpectedly when an object
becomes unreachable unexpectedly (but correctly); for example,
see my answer to this question
finalization can easily break subclass/superclass relationships
there is no ordering among finalizers
a given object's finalize method is invoked at most once by the JVM, even if that object is "resurrected"
there are no guarantees about timeliness of finalization or
even that it will run at all
there is no explicit registration or deregistration mechanism
The above are difficulties with the use of finalization. Anyone who is considering using finalization should reconsider, given the above list of issues. But are these issues sufficient to deprecate finalization in the Java platform? There are several additional reasons explained in the sections below.
Finalization Potentially Makes Systems Fragile
Even if you write an object that uses finalization correctly, it can cause problems when your object is integrated into a larger system. Even if you don't use finalization at all, being integrated into a larger system, some parts of which use finalization, can result in problems. The general issue is that worker threads that create garbage need to be in balance with the garbage collector. If the garbage collector falls behind, at least some collectors can "stop the world" and do a full collection to catch up. Finalization complicates this interaction. Even if the garbage collector is keeping up with application threads, finalization can introduce bottlenecks and slow down the system, or it can cause delays in freeing resources that result in exhaustion of those resources. This is a systems problem. Even if the actual code that uses finalization is correct, problems can still occur in correctly programmed systems.
(Edit 2021-09-16: this question describes a problem where a system works fine under low load but fails under high load, likely because the relative rate of allocation outstrips the rate of finalization under high load.)
Finalization Contributes to Security Issues
The SEI CERT Oracle Coding Standard for Java has a rule MET12-J: Do not use finalizers. (Note, this is a site about secure coding.) In particular, it says
Improper use of finalizers can result in resurrection of garbage-collection-ready objects and result in denial-of-service vulnerabilities.
Oracle's Secure Coding Guidelines for Java SE is more explicit about potential security issues that can arise using finalization. In this case it is not a problem with code that uses finalization. Instead, finalization can be used by an attacker to attack sensitive code that hasn't properly defended itself. In particular, Guideline 7-3 / OBJECT-3 states in part,
Partially initialized instances of a non-final class can be accessed via a finalizer attack. The attacker overrides the protected finalize method in a subclass and attempts to create a new instance of that subclass. This attempt fails ... but the attacker simply ignores any exception and waits for the virtual machine to perform finalization on the partially initialized object. When that occurs the malicious finalize method implementation is invoked, giving the attacker access to this, a reference to the object being finalized. Although the object is only partially initialized, the attacker can still invoke methods on it....
Thus, the presence of the finalization mechanism in the platform imposes a burden on programmers who are trying to write high assurance code.
Finalization Adds Complexity to Specifications
The Java Platform is defined by several specifications, including specifications for the language, the virtual machine, and the class library APIs. Impact of finalization is spread thinly across all of these, but it repeatedly makes its presence felt. For example, finalization has a very subtle interaction with object creation (which is already complicated enough). Finalization also has appeared Java's public APIs, meaning that evolution of those APIs has (up to now) been required to remain compatible with previously specified behaviors. Evolving these specifications is made more costly the presence of finalization.
Finalization Adds Complexity to Implementations
This is mainly about garbage collectors. There are several garbage collection implementations, and all are required to pay the cost of implementing finalization. The implementations are quite good at minimizing the runtime overhead if finalization isn't used. However, the implementation still needs to be there, and it needs to be correct and well tested. This is an ongoing development and maintenance burden.
Summary
We've seen elsewhere that it's not recommended for programmers to use finalization. However, if something is not useful, it doesn't necessarily follow that it should be deprecated. The points above illustrate the fact that even if finalization isn't used, the mere presence of the mechanism in the platform imposes ongoing specification, development, and maintenance costs. Given the lack of usefulness of the mechanism and the costs it imposes, it makes sense to deprecate it. Eventually, getting rid of finalization will benefit everyone.
As of this writing (2019-06-04) there is no concrete plan to remove finalization from Java. However, it is certainly the intent to do so. We've deprecated the Object.finalize method, but have not marked it for removal. This is a formal recommendation that programmers stop using this mechanism. It's been known informally that finalization shouldn't be used, but of course it's necessary to take a formal step. In addition, certain finalize methods in library classes (for example, ZipFile.finalize) have been deprecated "for removal" which means that the finalization behavior of these classes may be removed from a future release. Eventually, we hope to disable finalization in the JVM (perhaps first optionally, and then later by default), and at some point in the future actually remove the finalization implementation from garbage collectors.
(Edit 2021-11-03: JEP 421 has just been posted, which proposes to deprecate finalization for removal. At this writing it's in the "candidate" state but I expect it will move forward. The deprecations added by this JEP are a formal notification that finalization will be removed at some point in a subsequent Java release. Perhaps not surprisingly, there's a fair overlap between the material in this answer and in the JEP, though the JEP is more precise and describes a moderate evolution in our thinking on the topic.)
(Edit 2022-04-04: JEP 421 Deprecate Finalization for Removal has been integrated and delivered in JDK 18.)
