Yeraze's Domain 3.0

DALL-E

Published by Randall Hand on August 20th, 2022.

So last week I finally got into the Dall-E 2 beta.. It works similar to MidJourney, you give it a text prompt and it generates 4 high res images. Unlike MidJourney there's not the annoying 4-square and then requesting an upscale, you basically start with 4 upscales. However the results from Dall-E look much more like attempts at Photographs, compared to Midjourney's oil canvas aestheic.

For example: "dragon in times square"

from DALL-E:

from Midjourney:

Just don't ask it to do anything with human faces.

DALL-E: "absence"

MidJourney - Art by AI

Published by Randall Hand on August 8th, 2022.

The headline is a bit clickbait.. As a purist, I say this isn't "AI", it's "Machine Learning".. sophisticated deep networks trained on millions of public domain images that can retrieve and combine components to make new conglomerate images.

I've been playing with MidJourney. I don't think it has any actual understanding of what it's doing.. for example, given the prompt "bruce campbell", you get this:

An impressive likeness for sure, But some of these look almost like exact stills from Promo material from some of his work. Similarly, given my name "randall hand" it seems to have been trained on some of my public imagery as well. The image below took a bit of triage, but it's the result for "randall+hand profile photo photorealistic"

I would love to know more about what data MidJourney is using for their training set. I would bet they're absorbing a few public domain datasets of public imagery and classical art, and then probably a mountain of data from sources like InstaGram. This would give them image itself, and the textual description provided by users (same as their current prompt system).

The MetaVerse: Applications

Published by Randall Hand on May 15th, 2022.

So last time we spoke about one of the core pieces of MetaVerse: The ability of a device to recognize where it is, and have information on what possible augments are available to it. That's well and good, but how do we build these augments in a flexible and constructive way that can respond not only to the environment, but to the user?

Well, there are lots of companies throwing their weight around in this space. Depending on who you follow on twitter, you may believe that SnapChat lenses are the future of XR.. Or maybe it's the systems coming out of Niantic. Or, like so many people, you believe it's Unity or Unreal, the gaming engines. It makes sense in a way, they already have all of the 3D Rendering and Physics capabilities, so why wouldn't that be the natural solution?

In short, it's too hard. You might think to yourself "Unity? Hard?" but let me continue.

The MetaVerse has to be built on a set of open API's. The language and tools used to build these systems can't be owned by any one company, no more than there's a single source of C++ or Javascript tooling. There are big players and tiny players, and all of them work for various use cases. It will be the same for MetaVerse.

There will be experiences that remap the world around you to look like Hogwarts, where you might find a dragon or a dementor around every corner, and for those experiences maybe Unity is the right choice. But there will be equally many, probably even more, experiences that are much more mundane and "usability" focused. Where, in your location, can you knock off your shopping list? What about the bus or subway schedule? Maybe you just want to order some coffee at StarBucks using their MetaVerse ordering system, and get directions to where you can pick it up. Or maybe you just want a little AI-driven bot to follow you around and be available, like a virtual Alexa or Siri.

Unity isn't the best solution for these systems. These systems need to be lightweight and responsive to the user and environment. There's a whole world of new systems coming in the near future for "intelligent environments" that we have only just begun to see. If you pop open a "Window" in MetaVerse contexts, you don't want it to appear Z-fighting with reality, or behind physical objects and leading to migraine-inducing focus problems. Just like Windows went through a few years of learning all about smart window placement, we're going to deal with the same thing in XR content, and no application developer is honestly going to want to deal with it. It's an OS & "Window Manager" problem, and should be dealt with as such.

In addition, there's a whole new set of interfaces required in XR space that we don't have good solutions for today. Some experiences will be user-triggered, doing smart things like starting when you get close enough, or when they detect they "have your attention". The exact logic of how these work will be tricky, but requires a standard set of triggers that can be shared between Devices, Applications, and Cloud Services.

I think it's no coincidence that JavaScript frequently tops out lists of the most popular programming languages. It's ability to run in almost every environment, and the wide collection of tooling and libraries, all make it a great swiss-army-knife of a language. It also puts it in a unique place to serve XR content. The "MetaVerse React" Framework would be something that could be deployed to a serverless infrastructure, connected to a Map Server, and have a combination of Backend and Frontend code to build an experience.

Imagine this: As a developer, you get an API key from your chosen Map Server. You load that API key into your node+React application. Now your application can get webhook callbacks and launch when a user enter's the "interaction zone" around your app, and serve up content. You application can also post limited information back to the Map. Not physical map details, but metadata tags like basic State information. Some of the information will be global ("App in use", "Down for maintenance") and some of it will be instanced (What the user is currently looking at, states of the UI for their experience), and the app has the ability to decide how it handles all of this. Some information will be personal, some will be shared. Sometimes I hit a button and only I know, sometimes I hit a button and everyone sees it change state. If it's a cart checkout app in a store, probably I'm the only one that sees all the details, but if it's a virtual Dartboard then everyone should be able to see what's going on. JavaScript and the related frameworks are well suited for this, but this is all build on Open API's so there will always be people building similar experiences in other frameworks (Unity, C, etc).

JavaScript as a 3d rendering platform will seem foreign to many people, but you have to realize that the bulk of MetaVerse use cases won't actually be 3D. It'll be what we do today in 2D, in 3D. By that I mean effectively "floating panes of glass" with minimal 3D effects. Virtual posters and checkout lines in a store, enterprise dashboards and oversight use cases, even most training and medical use cases will be flat panes of glass with the occasional arrow into 3D. There will be a few companies that take it the extra mile to fully 3D augment things, and they'll be successful. But we'll find the market flooded with lower-fidelity experiences that provide sufficient value and meet people's needs.

I estimate it will take about 10 years in this space, only then will we start to see the really wild 3D stuff start. Look at the Smartphone industry: If we use Apple and Android as the primary example of smartphones.. They all basically started as glorified web browsers. In fact, a "good web browser" was a primary selling point of the original iPhone. When apps first became a thing, most of them were just packaged websites. There were a few exceptions, Angry Birds and the like. But 95% of the content, and user engagement, was in Twitter, Facebook, note taking apps, mapping apps, and all of the other things that people actually needed. Games are a good entry point and make flashy advertising, but you have to provide value in order to maintain engagement.

And User Engagement is going to be the key metric for MetaVerse. How do you get users to use your experience? And use it often? Because by using the experience, you're collecting data not just about their shopping habits or their browsing history, you're collecting data on The User. Their Home, their workplace.. How many square feet is it? How cluttered is it? What brands of appliances do they have? What colors do they like? Are they sluggish or hyper? In their room, what do they look at regularly? This is why companies are pumping millions of USD into VR, AR, XR tech companies. This is the dark secret of MetaVerse that gets companies and advertisers excited, and it's what they're now working to build programs for...

How do they convince you that you need this?

The MetaVerse : Maps and Location

Published by Randall Hand on May 8th, 2022.

So to carry on from the last article, how do we get the MetaVerse up and running? There's a list of components to build before we can really have it:

A way to correlate a Physical Location to an Internet Server.. This is required to find the host for the tracking map, and to coordinate servers that could provide augmented content.
A server architecture for hosting this map. It has to support special protocols to provide regional map information, because the full map is probably too big so we need the ability to chunk spatial information up and down in something like Octree arrangements. It also has to support user contributions for map updates, which the server will consolidate into the canonical truth map.
API Contracts for how this server will notify both users and relevant servers/applications of important map updates. E.g. New areas are discovered, areas change significantly, or users enter/exit an area of interest.

So let's start with the first one: How do we correlate a Physical location to an Internet Server? This internet server serves two primary functions:

Providing the map that can be used for tracking, and for rendering (for remote attendees)
Providing coordinate information for augmented content.

We already have services like DNS that can convert a readable name into an Internet Server address.. So how do we get geospatial location into this? Things like GPS can give us coordinates, but there's so much noise and so many digits that trying to map it would be difficult. That is, if it wasn't already done for us.

Introducing Google Plus Codes. A free opensource system from Google that can turn a Physical Address (Street Address or Lat/Long) into a short 5+2 format code. For example, my local WalMart is 8P3F+Q7 (Among many others). Google Plus codes get you a space about half the size of a basketball court, which is plenty for any location tracking system I'm familiar with.

So what we need is something that can calculate the Plus Code for your location, and reconcile that Plus Code against a list of Map Servers to get you started in your space. Note that not all Map Servers will be available to all users. Some map servers will be restricted behind company firewalls or authentication requirements and such, due to privacy or company IP concerns.. That's fine, we want to support that.

This same system can also provide information on servers that provide content for the area. These servers would effectively be webservers for Augmented content, running applications and providing content anchored against their corresponding Map server.

Some important points to remember

Just because you're in the right plus-code, you may not be able to use any provided map. A plus code could cross a boundary between an indoor and outdoor space, or your Geolocation could be inaccurate and provide you a bad map server at first.
Your device needs to be able to handle multiple maps. If you're outdoors but walk into a location, you need to be able to recognize that transition boundary and handle it. In addition, a space may have an owner-provided map and then there's a separate "community built" map, which are slightly different. I expect this will happen a lot at first in retail locations, where venue owners provide high-resolution maps built from Lidar or other specialized hardware and reject incoming updates, so there will be lower-resolution but more frequently updated community built maps. Think the early days of MapQuest vs OpenStreetMaps.
In a true MetaVerse scenario, users need to be able to handle multiple maps and multiple content providers simultaneously. E.g. if I walk into a Walmart while I'm waiting for my Uber to arrive, I need both my Uber Status and the relevant store content simultaneously, which will come from 2 different providers and reconcile to 2 different map.

So what does this look like to a user? You walk into a space and, either automatically or explicitly via a "Find a MetaVerse" function, geolocate your position roughly and find your Plus Code. You then communicate with a centralized Spatial Mapping Index Server to find a list of Map Servers available for your location. Your device can then either prompt you to select one, or perhaps it just tries them all in parallel to see which ones work. Once it's verified it has access to the map servers (remember, firewalls & authentication requirements) it looks to see what content servers are available for the functional maps. Once the list of available content is compiled, it can either be presented to the user ("Select your Metaverse Content"), or possibly automatically enabled based on user preferences (e.g. I'm a frequent Lyft user, so I turned Lyft content to "Always On").

There's a whole host of fine details in here too:

Who runs these Spatial Mapping Index Servers? And how do you verify them as safe? I assume a group like ICANN will have to be stood up to manage the "big ones", and several of the big internet companies will join in and offer alternatives and replicas. Individual companies will stand up their own, just like they do today for internal networks. Fun thought: Can you imagine what Split Horizon will look like for this?
Access Controls - How do you manage which users have access to which maps? And which content? And which content has permissions to which maps? And which users's information?
Verification - How do I verify that content coming from a server is actually tied to a specific vendor? There are things like SSL Certificates and such that help here, but they need to be extended a bit.
Map ownership - How do I know that a map server is actually providing a useful map of a space? Things like Chain of Trust and rating/expiration systems can help here.

All of that together is enough to get a user into a space, with maps they can use for tracking and basic information on what content is eligible for them to view... Once they have that, how do they actually get content? And how to developers create it? We'll save that for next time...

The MetaVerse

Published by Randall Hand on April 14th, 2022.

Seems all the news these days is about the MetaVerse. Facebook, Niantic, and others keep talking about it. FortNite is the MetaVerse, RecRoom is the MetaVerse, PokemonGo is the MetaVerse, people just won't stop pointing and things and going "That's the Metaverse!" ... It's as bad as a 3 year old pointing at every animal on four legs and going "DOG!".

Of course, it's not fair to start an article like that without planting my own flag on what the metaverse is. So what do I think the MetaVerse is? First off, it's not a product from any one company. Just like Microsoft doesn't own Word Processing, no one company can own the MetaVerse.

The MetaVerse, or the "MagicVerse" as we used to call it at my previous employer, is a set of open standards that can be implemented by multiple vendors and consumed by multiple platforms. The MetaVerse is a means of distributing Artificial Reality content to consumer viewing devices. Some of these devices will be head-worn AR and VR headsets, some of them will be hand-held tablets and phones, and some of them will be fixed Desktops or Kiosks. The content that's displayed will sometimes be augments onto an existing physical place, and some will be fantastical otherworldly locations that simply cannot exist in reality. With proper infrastructure, all of them can be enjoyed by all people.

In a MetaVerse enabled world, physical locations will be able to be augmented in both entertaining and useful ways for consumption on a wide variety of devices. The part nobody wants to talk about is that this already happens today, and in the most mundane of ways. Have you ever been in a big retail store, for example Home Depot or Walmart, and you load up their app on your phone and it detects which store you're in, and then can tell you where in the store that item can be found. Welcome to the MetaVerse, it's already here.

But that's not the sexy example people want to see. With a bit more effort, the same inventory system could be correlated to a 3D map of the store. Then, in an AR Headset (like a MagicLeap, NReal, or Hololens) you could do the same thing, but have a visual overlay in your field of vision, possibly even with some value-add features like a navigation line to the products location. Lastly, the store could perform a high-resolution scan of their location and present it in a VR Headset for someone to view remotely. All of these lie on the spectrum of MetaVerse content, each leaning in to the best features of those platforms. Nobody wants to wave a phone around at arms length for any period of time, and nobody wants to try and decipher Aisle and Bin numbers in a headset.

The only true MetaVerse realization will come once some standards emerge so that all of these platforms can use the same base map and location positioning system. Basically, how do all of these devices know where the checkout counter is at? No physical location wants to manage multiple maps, one for each vendor. It would be the same as VHS vs BetaMax, or BluRay vs HD-DVD, eventually one would dominate and the others would bend to adopt it. There must be a way to generate a single map of a space that can be both a good CV/SLAM Representation of the space to use for location work, and a good photorealistic representation for rendering work. Once this exists and tools are created to manipulate it (even to create it from scratch for those otherworldly location), then the MetaVerse will bloom. Once a location owner can create a map and then hire an overseas developer group to build an experience, without ever actually setting foot on site, that'll be the first steps. Once a high schooler can download a map of a public space and start augmenting it in exciting new ways from the comfort of their bedroom, that'll be when we start to see the truly amazing and bizarre flourish. And once these maps and their relevant content become the newest DNS record extensions, we'll finally have ubiquity.

What the MetaVerse needs is a LAMP stack. DNS Records with geoposition relevance pointing to elegible MetaVerse content servers, searchable by Google and other automated systems, and with API's to allow programmatic access to the space and the space content from non-augmented systems. Then, when I enter a physical location I get a notification on my MetaVerse enabled devices "Would you like to join the Metaverse?", and it automatically knows what code to download and servers to connect me to. And if I'm at home and want to join a MetaVerse remotely, I can fire up GoogleVerse and find a MetaVerse of interest to join and explore. While I'm there, algorithms are continually refining the content and using information about me to make recommendations or alterations. Interested in sports? Look over here! A bit short? Let me show you what's on the top shelf automatically. Last week when you were here you were looking for Ceiling Fans, would you like to see a new model we just got in? These changes are not globally visible to everyone, they're all a tailored experience just for me.

The MetaVerse is not any single app, it's a platform for distributing content to multiple viewing devices that can then independently determine how to view it. And you know what that sounds like? The Web.

Lacrosse Gear Drying Rack

Published by Randall Hand on April 13th, 2022.

So our son plays Lacrosse.. He plays a LOT of Lacrosse. And we live in South Florida, so he plays sweaty lacrosse. As the parent of any teenager will agree, sweaty boys reek. Add in lacrosse pads, and leaving them in a gym bag for days or weeks at a time, and the stench can kill most mortal men.

So how do you deal with this? The jersey and shorts you can wash, but the pads? What do you do with the pads? Gloves? Shoes? It all reeks, and the stench clings to things like animal hair during shedding season.

After some research, the easiest solution seems to be basic fabric refresher sprays and sunlight. Lots and lots of sunlight. Hockey players deal with a similar problem, and there are lots of designs online for gear drying racks for hockey gear. I found one I liked and built the same.

Assembly took all of 30 minutes to put together. It's all built from half-inch PVC (although seeing it now, I should have used 1-inch PVC, this thing sways and flexes an uncomfortable amount). The whole thing was $63 in parts from Home Depot, and that included a $14 pipe cutter and $10 in PVC glue that I only used a fraction of. I bought 10 pieces of 2-ft PVC sections, 10 1/2 inch PVC caps, and a bin of T connectors.

It works pretty well. The only drawback of this: Getting him to actually use it.

DIY Slide Out Pantry

Published by Randall Hand on April 13th, 2022.

In our kitchen, we have a small pantry next to the fridge that's 5 feet tall, 2 feet wide, and 2 feet deep. It holds a surprising amount of stuff, but it results in so many things getting lost. Being 2 feet deep, you really only see the front 3 or 4 inches, and things get slowly pushed towards the back as new things go to the front. Every 6 months we throw away almost half of what's in the pantry, and the cycle repeats.

So a few weeks ago I finally convinced my wife to let me convert it into a side-by-side pullout. The project itself was simple and really only took a weekend. I used 2 sheets of 3/4" Birch Plywood ($75/sheet), and two sets of heavy duty pantry slides from Rockler The carcasses are simple boxes with dado'ed slides for the shelves, glued and screwed together.

The end result has greatly improved the usability of our pantry, but it's not perfect. Problems so far:

Items slide around a bit, causing things to shift and occasionally fall of. We've mitigated this by placed small bins and shelf-liners in.
They don't slide very well. I thought this was the weight, but particularly the right door doesn't slide smoothly. I can't imagine we have over 130lbs of stuff on a side, but it consistently locks up at the halfway extended point. We haven't found a good solution for this yet.

All said, we're happy. I'm probably going to make some more minor changes, but I'm calling this one a success in our book.

woodworking

Solar Charging details

Published by Randall Hand on October 6th, 2021.

So, one of the first problems we ran into with the RV was a steady drain on the engine battery. One of the main reasons for getting the RVwhisper all setup was to monitor this data remotely and see if it was real or just typical lead-acid battery flakiness.

After replacing the batteries to eliminate old batteries as a cause, I started collecting data, and noticed the following pretty quickly. Here’s a graph of the House battery (2 6V Batteries in series) and the Engine battery.

(Green in House Battery, Red is Engine Battery, grey bars indicate nighttime so you’re seeing the voltage over about a week).

Nothing was on or running during this period, the RV was just sitting in storage and the RVWhisper & Modem were plugged into a portable battery (another article). You can see pretty clearly that there’s a substantial drain on the engine battery here, dropping about 0.3V in a week compared to the House Battery.

To some, this is a known problem. Lots of things in the RV continue to run when it’s "off", things like smoke detectors and such. To avoid showing up to a dead battery, many people recommend disconnecting the battery. That seemed a bit extreme to me, so I moved onto the next solution: Solar Suitcase.

Solar Suitcases are popular in the RV world for this very reason: Small portable solar panels with the charge controllers and all integrated, that you can hook up via giant gator clamps to your battery and provide some offset current during the day. After a bit of research, I went over board and bought a big 200 watt/20 amp Renogy panel.

With that hooked up, I’ve never had any more battery problems. But I did notice a new interesting quirk in the charging graph, that looks like this.

Every day there’s a swift uptick in voltage to a peak just under 14.5V, then it falls back to just under 13.5V to spend the rest of the day, before falling back down to 12-and-change at night. What’s going on here?

I suspect that the first 14V spike is the actual charging step. Measure a running car battery with the alternator going and you’ll read something similar. This is when it’s actually pumping a fair bit of juice into the battery to charge it up. Given that I went overboard with a big 200W solar panel, it doesn’t take long. Then it drops to this 13.5V range, where the battery is basically "full" but it’s still absorbing a tiny bit of power to counter whatever drain is coming from the RV. Then it simply quits charging overnight when there’s no sun for the solar.

Seems legit, but can I monitor this more usefully? A bit of python and SQL query later, and I’ve added a new tool to the RVwhisper-Monitor package, solar.py. You can run it independently or have it added to the output of the slope tool, and get a chart like the following.

This shows the hours of sunlight in the day (actually seconds, I need to fix that sometime) as the faded red bar (Yes, the days really are getting shorter now). It also shows the time spent in that 14V Peak range (in green), and the 13.5V "sustained" range (in blue).

You can see that on days after a trip when I hook the solar panel back up for the first time in a while, there’s a lot of green. That makes sense, as it has to recharge from the few days of running the battery down. As things recharge over the first day or two, you see the blue begin to dominate as it’s now just having to recharge the continuous draw. And given the size of this solar panel, I can also power my Netgear Nighthawk & the RVwhisper module directly from the engine battery.

Hope you find this useful!

So I bought an RV....

Published by Randall Hand on October 4th, 2021.

A few months back, Laura talked me into buying an old RV from a friend of ours. WE thought it would be a great way to get our elderly Mother-in-Law back home to Mississippi to visit relatives. It's been an interesting few months.
So the RV in question is a 1994 Fleetwood Bounder 34C. Chevrolet engine, 34 feet long, and almost entirely original. Original carpet, original wallpaper, original electrics. Needless to say, she's needed a lot of work. We've been taking it out regularly to work on it. So far we've replaced the flooring (some old stained carpet, switched out for vinyl flooring), replaced the wallpaper, repainted the cabinet, replaced the power inverter, replaced the water pump, replaced the awning, replaced the AC Compressor... It's been a big project.

Over the course of all this, it's been important to monitor all the variables of the RV and see if things are improved or getting worse. It didn't take long to stumble across a product called RV Whisper

The RV Whisper is a low power communication devices that runs a local webserver and low-power communication network that can talk to a wide variety of sensors. I bought some temperature sensors (to monitor the temp inside the RV, as well as the fridge and freezer to make sure they work), door open sensors (To work like an alarm for the front door, and to allow me to correlation temperature inside the RV to the opening of the door), and some battery sensors for both the Coach & Engine battery (the batteries were one of the first main problems we tackled).

The device generates a weak wifi network that you can connect to directly to monitor the device, or you can connect to a larger Wifi and the RVWhisper company offers (for a small annual payment) a reverse proxy from their website to your device. They don't store any of your data, which is nice. However, this also means that if you have an interruption in your network then you can't get any data.. Not even historical.

Me being me, I wanted something a bit more robust.. And while I'm at it, if I could get some more sophisticated visualization and analysis I might be able to get more information on what's going on inside the RV.

Introducting "RVWhisper Monitor", freely available on Github.

I'm running this at home on my Raspberry Pi. Every 30 minutes it pulls an hour of data (the overlap helps with small network problems) and stores it locally in SQLite databases. There are also a few scripts there to visualize the data as static HTML files uses Chart.js for some basic interactivity. Check out an example here.

Once the data is stored locally, I can start calculating some more advanced visualizations. The most important one I've come up with so far is the Battery analysis visualizations that show the slope of the battery state, as well as things like min/max/average over select periods (daily). You can see some of those visualizations here.

I'm still working on it. It's been a great help so far, particularly since I'm forced to park my RV at a storage lot a few miles away from home. With the help of a decent solar panel to charge up the engine battery I can keep the Nighthawk modem and RVWhisper unit powered, and then monitor it from the comfort of my house between visits.

If you have any ideas or recommendations, don't hesitate to let me know!

More Audiobook Recommendations

Published by Randall Hand on November 19th, 2019.

I’m really enjoying audible so far. Between the one credit per month that comes with the subscription, and the 2 free Audible originals you get each month, I find myself having no problem keeping myself busy in the commute.. So i thought I would share some of what I’ve been listening to lately:

Feathers & Fire series: Books 1 -3 - Just finished book 1 this morning.. If you like Supernatural or Constantine, you’ll enjoy this.
The Institute by Stephen King - It’s Stephen King, need I say more?
Carnival Row: Tangle in the Dark - I’m told it’s a prequel to the new Amazon Prime show.. It was interesting, albeit a bit of an unsatisfying conclusion (presumably to lead you into the show)
The Home Front: Life in America during WW2 - Podcast documentary on the untold stories of life back home in the US during WW2. Lots of interesting insights, complete with real interviews and recordings make both recently and during WW2.
Mystic School of Musicraft - a Harry Potter-esque story of a girl who discovers she can do magic and joins a prestigious school after the untimely and mysterious demise of her parents. A bit young-adult, but fun and actually includes the corresponding music in the background.
The Adventures of Tom Stranger and A Murder of Manatees - a hilarious pair of books narrated by Firefly’s own Jayne Adam Baldwin, in the vein of Hitchhiker’s Guide to the Galaxy.
The Singularity Trap - SciFi story about a mining group that stumbles upon the secrets of an ancient alien race
The Man who Knew the Way to the Moon - The real life story of John Houbolt, one of the minds behind the first successful moon landing.
Every Tool’s a Hammer - Adam Savage’s rules of creativity
The Dispatcher - a short SciFi story about a world where Murder becomes impossible as people just "re-awaken" in their home when they meet untimely ends, and the loophole.

View More Posts →