Add calculate tokens helper

[jodendaal]

Add method that will calculate tokens a request will use for text completion and chat API.

This is useful when consuming the library in an application and needing to truncate prompt.

Help avoid each implementation using a different calculation.

[Alumniminium]

That would be very helpful. I keep getting exceptions because the input size + the maxtokens param exceed the models max token count.

[LyleRolleman-plst]

Bit late to the party, but you can use TikTokenSharp for this (port of the tiktoken python library)