I would have expected both pipelines to be equivalent.
They're not:
'foo', 'bar', 'baz' | write-host
It is the pipeline-based equivalent of the following (equivalent in ultimate effect, not technically):
foreach ($str in 'foo', 'bar', 'baz') { Write-Host -Object $str }
That is, in your command Write-Host receives input from the pipeline that implicitly binds to its -Object parameter for each input object, by virtue of parameter -Object being declared as accepting pipeline input via attribute [Parameter(ValueFromPipeline=$true)]
'foo', 'bar', 'baz' | write-host $_
Before pipeline processing begins, arguments - $_ in your case - are bound to parameters first:
Since $_ isn't preceded by a parameter name, it binds positionally to the - implied - -Object parameter.
Then, when pipeline processing begins, pipeline parameter binding finds no pipeline-binding Write-Host parameter to bind to anymore, given that the only such parameter, -Object has already been bound, namely by an argument $_.
In other words: your command mistakenly tries to bind the -Object parameter twice; unfortunately, the error message doesn't exactly make that clear.
The larger point is that using $_ only ever makes sense inside a script block ({ ... }) that is evaluated for each input object.
Outside that context, $_ (or its alias, $PSItem) typically has no value and shouldn't be used.
While $_ is most typically used in the script blocks passed to the ForEach-Object and Where-Object cmdlets, there are other useful applications, most typically seen with the Rename-Item cmdlet: a delay-bind script-block argument:
# Example: rename *.txt files to *.dat files using a delay-bind script block:
Get-Item *.txt | Rename-Item -NewName { $_.BaseName + '.dat' }
That is, instead of passing a static new name to Rename-Item, you pass a script block that is evaluated for each input object - with the input object bound to $_, as usual - which enables dynamic behavior.
As explained in the linked answer, however, this technique only works with parameters that are both (a) pipeline-binding and (b) not [object] or [scriptblock] typed; therefore, given that Write-Object's -Object parameter is [object] typed, the technique does not work:
# Try to enclose all inputs in [...] on output.
# !! DOES NOT WORK.
'foo', 'bar', 'baz' | write-host -Object { "[$_]" }
Therefore, a pipeline-based solution requires the use of ForEach-Object in this case:
# -Object is optional
PS> 'foo', 'bar', 'baz' | ForEach-Object { write-host -Object "[$_]" }
[foo]
[bar]
[baz]