I like you are having success here, and the key item "natural communication lines" which I very much agree with. We will always deliver conway's law.
I am curious on 1) the size of your teams, and 2) your opinion on how this would work with E2E product domain based teams (mixed skill sets) vs skills based teams e.g. User engagement team (front and backend devs covering sign on, emails, etc) vs Backend engineers (maintain all backend systems, etc).
If the browser uses WKWebView or UIWebView, you're right, you can "mock" an API by injecting JS, and through tunnelling you can recreate native functionality. Not if its SFSafariViewController. As someone who has done a lot of fiddling with JS to Obj-C/Java tunnelling for iOS and Android in production, it can be prone to errors and/or get your app rejected. Browser makers probably do not see the commercial value in going down this bespoke path.
Good question, still a "green" technology but I have had great successes with it for the past 4-5 years in production (ASM.JS before widespread WASM compatibility).
"promising that we'll be able to build web apps in any language" is not how I see WASM, nor is it really used in this way outside of transpiling Unity3D/Unreal games (this may be the one area there is an exception). I use it to transpile C++ to WASM libraries used within React apps, Edge Lambdas, and Node.JS servers. Primarily down to 2 reasons: speed and efficiency. WASM unlocks excellence and resources in other disciplines/languages such as AI and AR tool chains, Engineers, OpenCV, etc. When used like this, it is outstanding.
What WASM will never be good at is being used for the whole experience. You lose the semantic web, and/or accessibility tooling. Web has some outstanding guidelines and frameworks to help the impaired, screen readers and the like. Using WASM to pump a native app into a HTMLCanvasElement will lose all of these advances, therefore, WASM shouldn't be used for this use case (outside of games). Like all tooling, there is a time and a place to use them.
Below are a few links which use WASM in production:
iOS and Android don't have the same camera picker like Chrome on Windows 10, even then it is difficult to always override. I would recommend using enumerateDevices to allow you to pass the deviceId when firing getUserMedia. Not a lot of work. Implemented this a few times for cross-browser WebRTC experiences. A pro-tip is use getUserMedia with audio and video constraints set to true. This forces the permissions pop up to appear. You can kill the feed once the user has accepted the permissions, then ask them what device they want to use, and then fire getUserMedia with the correct deviceId. This is a [nice?] hack as some browsers don't allow you to use enumeratedDevices until after getUserMedia has asked you for the permissions. e.g. Safari...
Good question, I would say neither, this responsibility is for the browser and native apps you use which display web content (e.g. Chrome, Firefox, Facebook, etc). Not apple or the web app developer. At most OP should include https://www.npmjs.com/package/webrtc-adapter as part of their polyfills to aid cross-browser support/cater for WebRTC nuances.
WebRTC has been in iOS Safari for quite a while. However, all other apps and browsers were limited to what was in offer within WKWebView. From iOS 14.3 beta 1 onwards WKWebView gained getUserMedia/WebRTC features. It is now up to the browser and apps you use to make sure they have implemented their web views correctly and allow the correct permissions for WebRTC to work.
Used to live around the corner. Nice owner, good food and a great beer selection. I now [Ironically?] live in the states and miss that pub quite a lot.
To allow surgeons to interact with my software within an operating room without the need for an assistant (to remain sterile).
Interactive retail displays outside the store. Users can interact with augmented reality displays and visualize themselves wearing the store products and/or to play a game to win prizes, etc.
Problems encountered:
Hardware adequate for long experiences e.g. Microsoft Life camera freezes after a few hours. Finding a device which can run 24/7 is a problem. Then once you found a good device you need to understand the risk of it being pulled from the market e.g. Primesense, Kinect, Intel RealSense (pulled and replaced by a new product and SDK, etc).
If a depth camera is used the type of bulbs to sunlight can interfere with tracking accuracy. If RGB is solely used then I am curious to see how well it works with various skin-tone in different lighting conditions and complicated backgrounds.
The "heavy arm/hand" problem. Try lifting your hand for 5 minutes and not putting it down. Users can be fatigued very quickly with a gesture based UX. Most products are not designed for this interaction.
In terms of Zesture:
The website is clean, to the point, great starting point. However I would like to:
- See an Enterprise license for long term support
- Know how well it benchmarks against other SDKs/hardware solutions which achieve the same effect
- Patents, does this infringe on other proprietary innovations? (do you have patent troll insurance?)
- Guidelines for the best experience, e.g. distance from the camera if you were to use gestures to control a presentation
- Roadmap, where are you going next?(FYI I am looking for a new way of hand based gestures which can be deployed via WebRTC and WebAssembly for interactive web based experiences :) )
This is good advice. I would also extend this and write out an FAQ / stackexchange for the next engineer at your company who has to go through the same learning curve.
I am curious on 1) the size of your teams, and 2) your opinion on how this would work with E2E product domain based teams (mixed skill sets) vs skills based teams e.g. User engagement team (front and backend devs covering sign on, emails, etc) vs Backend engineers (maintain all backend systems, etc).