(Ahum, submitted too soon :))

See the screenshot. There are four of these strange edges. The selected face actually seems to be drawn incorrectly with respect to its actual edges.

There are worst bugs already listed about overlapping geometry.
I am not sure if this case of a shell should be considered as a duplicate or a different case.

Anyways, it is just a wrong choîce to triangulate an Ngon. User can easily fix this by adding a triangulate modifier after Boolean modifier or on Cylinder operand.

This is not really related to booleans, it's more that quads always tessellate first to third vertices.
This has been reported before and we've closed as known-limitation/behavior.

This was because we used to use GL_QUAD (in 2.8x this is no longer the case), so the order of tessellation wasn't defined,
In fact on some ATI cards this would show correctly.

Since 2.8x defines tessellation, it's probably best we support concave quads (as we do concave ngons).

Submitted D3052 to support this, would like to get feedback from @Bastien Montagne (mont29) since this has implications elsewhere (exporters for eg).