Is it possible to build a fully customized Session State Mode instead of using Inproc or SQLServer?
Yes, absolutely, you could write a custom session state store provider by implementing SessionStateStoreProviderBase class and then set the session state mode to this custom implementation in web.config:
<sessionState
mode="Custom"
customProvider="SomeCustomProviderName">
<providers>
<add name="SomeCustomProviderName"
type="YourNamespace.CustomSessionStateStore"
connectionStringName="SomeConnectionString" />
</providers>
</sessionState>