Came across this in-development app, that seems really clean. Is available for iOS/MacOS already. It will be available on PlayStore and Fdroid eventually, but there is an APK for those who want to try it out.
Github Project: https://github.com/avdept/JellyBoxPlayer
Screenshots
Not exactly what you’re asking, but I can recommend Audiobookshelf.