EDIT: There is a seemingly similar question already, but it doesn't explore what happens if you don't need to do an ARP request at all, which is what seems to be hindering the communication there.
The title says it all concisely. Is it possible? Does the higher-level constructs like an IP address even matter at this point, now that we know the MAC-address of the receiver, for all intents and purposes it might as well be in our subnet, right?
You mention that the sender "already knows" the receiver MAC address. That would only happen by statically configuring it, or by some other non-standard means. The same would apply to the IP routing table -- you might be able to "fool" it with some custom entries.
But doing all this may end up breaking normal IP communication. So you'd end up with two host that can only talk with each other and no one else.
– Ron Trunk May 05 '21 at 16:48