Home/Beyond/Title: Jewel® – A Semantically Intelligent Web Browser

A Major Breakthrough in Global Diversity

Written by Ken Granville

Avg. read time 11mins

Eliminating the Double Hop

You might ask, “what is the double hop?” It is what every person on the planet must endure if they want to learn how to code and do not speak the English language. A person who does not speak English must learn it (the first hop) before learning to write code (the second hop). As discussed here, almost all programming languages are in English.

This is a situation that practically every English-speaking person does not have to contend with. This is not simply an advantage that benefits English speakers, it is certainly a barrier to global diversity. There is a global shortage of programmers, which is essentially an enormous deficit in the ability to access talent around the world. Addressing barriers to learning and creating with technology, and sharing ideas are fundamental cultural diversity issues. This problem goes well beyond issues resulting from a shortage of coders. As discussed in our inaugural blog post on “The Essence® of Computing“, ideas are trapped.

In this blog, we’ll describe how Project Jewel can help the world to take a big step towards eliminating both the first hop and the second hop [no matter the subject or idea that needs to be learned/shared]. Before doing so, here is what the United Nations Educational, Scientific, and Cultural Organization (UNESCO) is saying about the plight of many language speakers around the world.

The International Year of the Indigenous Languages 2019 – IYIL2019

Languages are dying at an alarming rate. Between 1950 and 2010, 230 languages went extinct, according to the UNESCO Atlas of the World’s Languages in Danger. Today (2018), a third of the world’s languages have fewer than 1,000 speakers left. Every two weeks a language dies with its last speaker, 50 to 90 percent of them are predicted to disappear by the next century. According to Ethnologue, a catalogue of all the world’s known living languages, 1,519 currently living languages are at risk of death, with a further 915 said to be dying.

Humanity is at Stake

The United Nations considers language support such an important matter for humanity that it has designated 2019, International Year of the Indigenous Language. Indigenous languages in the State of Alaska are all at risk [read about it here]. As Gov. Walker stated, this is a “linguistic emergency.” I say it’s bigger than that. There are undoubtedly those who would speak to the business advantages of increasing numbers of people who can speak the same language. However, there’s an increasing number of advocates for endangered languages who argue that humanity loses when a language dies. Cultures lose the ability to contribute to humanity’s knowledge pool when their language dies. The stories of a group of people and what they’ve discovered of value [over centuries or even millennia] can no longer be added to the wealth of human culture as a whole. Who knows, maybe they hold the secret to saving all of humanity from a crisis.

Not for profit organizations like Wikitongues are doing great work to preserve at risk languages. Visit their website [wikitongues.org] https://www.wikitongues.org to see how capturing and archiving recordings of speakers of at risk languages is helping to fight language loss.

History suggests that language loss is inevitable, but it is appropriate to find out if the Internet Age is accelerating that outcome [by only supporting a subset of the total languages in the world] and what can be done to stop or reverse it.

Linkage Between Programming Languages and Opportunity

There are less than 20 million programmers in the world out of 7.5 billion people. To the overwhelming majority of people in the world, they don’t have the opportunity that comes from being able to create value in the digital age. They simply don’t know how to write code and the question is, why should they? For those who say the key is teaching more people to code, that view does not take into consideration the “double hop” challenge, nor does it acknowledge that humanity benefits the most from tools that are usable by everyone.

The true value of computing is what the machine does (behaviors), not the software (far too often the painstaking and error prone result of writing code). Wouldn’t removing this barrier open up a world of ideas that are currently trapped? The opportunities wouldn’t just open up for those who don’t speak English, it would increase opportunities for all of society, including English speakers who do not know how to code. For those who would say that this would be bad for software engineers, we have seen this type of change before.

What if this Blog Post was About Cats or Dogs?

What if cat and dog lovers learned that half the species of cats or dogs around the world would be extinct by the end of the century and that there was a relatively low cost way to prevent it from happening? It would be reasonable to expect that there would be a lot of support for that solution. We believe that Project Jewel addresses 3 key elements of the root causes of the world’s languages and cultures being at risk. They are; 1) Lack of language support. 2) Low speed Internet access. 3) Limited access to software engineers or inability to write code.

Project Jewel…A Precious Gift to Humanity

Humanity has benefited richly when greater opportunity to access gifts that had been limited to the few were extended to the masses. The expansion of travel to the masses created an explosion of explorers, commerce, leisure, and numerous other industries. Imagine what the world would be like if nearly everyone still had to walk everywhere and goods were transported by horse and buggies still [no automobiles, boats, trains or airplanes]. Although that might appeal to a few, the majority of humanity has benefited from these innovations and would consider it an apocalyptic event if modern means of transportation were no longer available. The closer the solution is to benefiting all of humanity, the greater the gift and impact.

Jewel is the world’s first Semantic-AI Powered Crypto Browser. Unlike web browser technology, a Crypto Browser uses next generation patented technology to instantly transform all signal data (text, photo, streaming video, web pages, sensors, and more) into 4 dimensional (3D plus changes over time) math equations. Signals become easily translatable (e.g. language to language, code to code), simple to transform (e.g. for higher quality than the original, etc.), and modifiable (e.g. using natural language to create/change behaviors of code). Jewel does all of this in real-time.

How Does Jewel Address Unsupported Languages?

In the case of text, the ability to dynamically transform existing low-quality font packages and other media, without pixilation and at the highest resolutions. Additional features of Jewel support more diversity in the global digital economy. A prototype of Jewel can quickly be made ready for release (see the 4 project phases below).

Phase 1 – Representation Written/Spoken

  • We currently support many text formats, from EBDIC, all legacy Unicode variants (UTF-8, UCS-2 vs UTF-16, etc.) to Big5, JIS, and IBM’s Code Pages are supported, with ligatures, proper Cyrillic italics, Semitic direction alternations, and other quirks are given support. We do this to be able to use historical and current data as we’ve seen how often certain standards endure and even thrive against the limitations of standards, such as the Han Unification complications or issues in Farsi layout (see: Harfbuzz). We have put a lot of effort into importing various Byzantine font composition, text layout, and quirks of representing writing on computers while being able to readily modify such rules to accurately portray a writing system. We can’t claim perfection, but we are designed for adaptation and the possibility for an end user, not a coder nor writing system designer, to make corrections, additions, etc.
  • Unicode 11, released on June 2018, has nearly 140K characters with 146 modern or historic scripts, symbol sets and emojis but it does not capture the many languages, nuances or styles in our world. We put years of effort in handling such scenarios, from Lost-Languages to Conlangs. We can allow language speakers or learners to create glyphs, describe rules (in an already known natural language, and later in the language they are working on) and ‘word-process’. By ‘rules’, we mean describing grammar, exceptions, conjugation approaches, spellings, colloquial phrase use case conditions, etc. Fundamentally this allows us to empower users to craft a series of valid vs invalid conditions and present that to any language user. The value is in capturing said rules for domains such as honorifics, gender, time, task-conditions, element counting, etc.
  • We currently support IPA (International Phonetics Alphabet) phonetics and 3 speech synthesis systems (HawkVoice, OS Native, which is currently the Apple voice set, and the TI-voices). We could add the data needed to express speech rules and get the expression going, with an ability to capture pure voice for phonetics and potentially preserve unusual vocalizations (bilabial splutters? soft golottal stop fades?, and things not adequately captured in today’s IPA).

Note: It’d be nice to add ability to send/receive from a USB Braille touch device as we expect it to be straightforward to add support (https://www.engadget.com/2018/05/31/microsoft-apple-usb-if-accessibility-braille-display-standard/)

Phase 1 Results:

  • Every written language – is supported, not just on the web. Every app that we build (starting with Jewel) and every app/service that we connect to (e.g. licensed for Chrome, Firefox, Safari, other native and web apps) supports every written language.
  • Extending hope to the rest – The foundation is established for subsequent phases that produce tools used to create new written versions of languages.
  • Greater access at lower cost – Low bandwidth locations (e.g. rural and developing world) benefit from ability to transform low res media (every web page, photo, video, etc.) into the highest resolution of the viewers’ display.

Phase 1 Duration:

  • 4 months to have a beta ready for release, initially on Mac OS
  • 3 months later for Android one-time port
  • 3 months later for iOS one-time port
  • 3 months later for Windows one-time port

Note 1: Order of one-time ports is negotiable.

Note 2: One-time port means we create a platform specific Agent (e.g. for Android, iOS, Windows) and every product (e.g. Jewel and every other product that we create) will work on every supported platform. Every product becomes platform agnostic.

Phase 2 – Understanding (Semantic/A step towards Semiotic Computing)

  • The basis of any word or combination of words or combinations of sentences of words is filled with objective and subjective meanings. We have a system to capture these meanings and find similar meaning expressions in other languages, styles of speech or personalization.
  • This approach not only enables users to preserve their subtler but critical meanings, but it also allows us to present additional info (as a hover, footnote, or other means) when translated into a different language or style of speech.
  • This semantic representation covers data (what happened, what is currently sensed or believed) and code (what to do, the instructions or recipe to follow and its exceptions, in natural language).

Phase 2 Results:

  • Understanding what is meant – we enable users of Jewel and any licensed app or service to capture meanings for better translation and user experiences.
  • Understanding what code to generate – when we know what the user wants to happen, we can generate machine instructions to make it happen [within rules, policies, legal constraints).
  • Understanding is sharable – gives us the ability to tell the user why code behaviors are what they are.

Phase 2 Duration:

  • 3 months to have a beta ready on all platforms (dependent upon porting projects in phase 1)

Phase 3 – Motivation of Use (Wantware) language as code

  • Building on the above ‘semantic’ capture, language users as well as learners have the incentive of being able to express, correct, and use a particular language or style as well as to monetize it by creating tradeable software behaviors, in their language and expressed in others.
  • While this idea deserves a couple papers of explanation and a demo, it can be distilled into ‘software behavior has value, if learning or using a language has the potential for revenue, it’s an added incentive to preserving, utilizing, and sharing’.

Phase 3 Results:

  • Create software behaviors – we enable users of Jewel and any licensed app or service to create software behaviors [without writing code] and facilitate changing them [within rules, policies, legal constraints], live.
  • Trade software behaviors – we enable users of Jewel and any licensed app or service to trade software behaviors [within rules, policies, legal constraints] and the recipient to incorporate them, live.
  • Monetize software behaviors – we enable users of Jewel and any licensed app or service to monetize software behaviors [within rules, policies, legal constraints] and the recipient to incorporate them, live.

Phase 3 Duration:

  • 3 months to have a beta ready on all supported platforms (dependent upon porting projects in phase 1)

Phase 4 – Expression (Emotional, artistic remapping of messages)

  • The MindAptiv Pipeline is built for transforming, compositing, and tweaking visual and audio signals. We are able to mix a variety of expressions to create a broad range of effects and interactive visuals, such as certain words having a colored glow that syncs to the mic so your voice or background music makes it pulse to the beat or embedding it into various materials like beach sand or clouds or filling words with ‘mini-words’ related to it via synonym or another language’s synonym for learning.
  • Our pipeline allows a lot of personalized expression to be created and shared.

Phase 4 Results:

  • Expression Creation – we enable users of Jewel and any licensed app or service to express emotion and artistic messages as software behaviors [without writing code] and facilitate changing them [within rules, policies, legal constraints], live.
  • Share/Trade Expressions – we enable users of Jewel and any licensed app or service to share/trade expressions [within rules, policies, legal constraints] and the recipient to incorporate them, live.
  • Monetize Expressions – we enable users of Jewel and any licensed app or service to monetize expressions [within rules, policies, legal constraints] and the recipient to incorporate them, live.

Phase 4 Duration:

  • 3 months to have a beta ready on all supported platforms (dependent upon porting projects in phase 1)

Note 3: Timing of subsequent phases may change subject to work priorities.

In Conclusion:

We invite others [including browser companies interested in a major advancement in global diversity] who are interested in helping us to achieve these goals:

  1. Every written language online within 1 year [subject to desires of indigenous communities]
  2. Written support for every spoken language [subject to community opt-in] online within 5 years
  3. Translating between every written language within 7 years
  4. Translating between every written language and machine behaviors within 10 years

This is a very ambitious undertaking, which is possible for the first time in human history. What a gift to humanity? We would love it if you would complete a short 9 question survey [here], which will help us determine support for Jewel. We will not collect any personal information during the survey.

Follow us on social media and our website as we unveil more potentially world-changing solutions.

Get ready to ‘create at the speed of thought®‘

Ken Granville & Jake Kolb

Cofounders of MindAptiv®