Your idea is partially true about using Web API on client devices but you cannot get rid of the “Client App”. The REST API is fine, depending on what technology you wish, it can be coded and hosted so the consumers will have access to it.
Imagine, you are developing a RESTful service and host the same in Raspberry PI first, and then develop a Client App that can consume the REST API, you are all set.
Please take a look into my project in Collaborizm for more understanding on API development. https://www.collaborizm.com/project/rk4T9qpGe