BUILD PIPELINE QUESTIONS
Q1: I am most concerned about security and vulnerability scanning that would take place during a normal software build/SDLC process. Generally, those things are hard requirements to operate in a commercial or government space. Can that be explained in any additional detail?
A1: If you only use wantware for generating source-code or an application that gets dropped into your normal process, then everything proceeds as you’d expect. If you wish to use Essence to run your products, you can incorporate many pipeline tools as E-Powers or replace them in wantware for better transparency, control, and improvements to your particular needs. This assumes you are running an Essence in realtime to edit, whether alone or collaboratively, or just to provide it to clients with limited control.
Q2: If I think about a Jenkins build pipeline for example: I would pull source code from version control first, how is version control handled if this thing is just changing that underlying source whenever I say something?
A2: You can use existing version control systems for your wantware & data/media-assets. We do this internally for multi-pronged backups. Note that the ‘wantware’ itself is not just ‘plain text words’, but for every term or after every section of words (not a grammatical clause, just clusters of association terms), we embed the ‘meanings affirmed by the user’ and the default-contexts that fill in needed extra meanings (whether it’s the visual interface meanings you saw in our Scoreboard App, or 3D simulation objects or data-type/database ones, etc.). So, what you’re ‘checking-out’ is usually ASCII-7, UTF8 or UTF16 text files which embed the ‘meaning mappings’ into the file. The embedding can be hidden, such as inside the ASCII-control-codes of the first 0…31 entries or the upper control bits of extended ASCII or UTF-user space. Or the embedding can be explicit using an escape symbol and either the Phonetic Symbols (PyDru), visual symbols (in HTML we could embed the glyphs as symbols with Base64 encoding or use Markdown w/ Pictures), or possibly user-defined abbreviations. However, that is not expected as a common use case and available primarily for MindAptiv validation/testing.
Q3:Does this work with an industry standard like Git or GitHub?
A3: I use Git regularly for our core Agent/Origin source code with a repo living on my LAN’s NAS. I have some E-Powers on GitHub for eventual 3rd party use. Think of that as a simple SDK & working-examples to expose your existing software or network services to wantware. However, if you use Essence as your runtime, it can track changes directly by who changed what & when in multiple timelines (forks with ownership/rights) and versions. We offer this for compression of delta-encoded changes as well as fast replay/undo when iterating. Ownership is important for collaboration, security, and successful concurrency scheduling and forms a key aspect of our internal version control. This approach, which is an Essence-Chronicle, doesn’t preclude any individual or organization from using GitHub or others. I would expect it as our approach is Peer-2-Peer networked, whether on a fleet of your own machines or to others across the globe. The encryption enforcing validity and the ledge with proofs of validity claims are provided as E-Powers and can be changed for each Asset and Owner as often as they like.
Q4: How would a development team of more than 1 work in this type of environment without working on top of one another?
A4: As is the theme, both existing commercial text-file merge tools, such as Git/Mercurial/SVN or manual ‘Diff’ & Edit, and native wantware-fork/merge work with multiple contributors. I use different machines & IDs to edit the same section of code on different cloud or NAS copies when I’m in my lab at a workstation (via text & direct wantware edit), my laptop while in bed or my phone (via text edit) while ‘anywhere else’. I either use text merge tools (Visual Studio Code, XCode, Unix diff & text edit) or an ‘in-progress’ native wantware tool (shows side-by-side changes for 1 to 4 ‘conflict-files’ and allows selecting the merge result). The key aspect is that multiple contributors to the same section rely now have ‘two-sources’ to work with for conflict detection, ordering, and resolution which are the raw-text-input (what you said) and the resulting, user-approved dialog-resulting meaning (meaning symbols that are mapped to the original words, but can be readily remapped to other words and statement-templates, as before, think of ‘rewriting a Madlib sentence.’
In the collaboration space, I find ‘real time’ collaboration, as you might do with multiple-folks-editing a Google Doc or Spreadsheet, quite valuable. The current state of the art is rather buggy in my experiences and by showing the choices-made (such as “why did that word disappear while we kept the other one, etc.”) to be valuable. We have an Essence-Power for resolving live conflicts and ordering changes which mostly follows the Conflict-Free Replicated Data Types approach. It is not complete but the architecture of “InfoSigns” makes it easier for us to handle editing as ‘Meaning’ instead of plain text or spreadsheet cells since we have more information for ‘invalid/not-possible’ than with raw data alone. Again, real time collaboration, whether on wantware-behaviors, or edit data-records or 3D-objects in a VR scene, all use the same path of having a stream of changes for a given version, timeline, and access-guard (rights & encryption).
Q5: How are variables declared and shared?
A5: We have lots of meanings to use to explicitly control access, storage location, persistence, records-of-change-vs-one-value-only, resolving-conflicts & protocols for Byzantine Generals-consensus, and chip-restrictions. However, if you just say I want a table of 1 billion numbers that go from 1 to 10 or just a single temperature value, we’ll take your request and map it to the best-guess meaning. So, the big array might end up using disk storage with a RAM page for most-recently-used if we choose x86 solution and it might be cycled into VRAM-memory-mapped tiles for an Apple M1 GPU. We can show all the details or very few pending the user’s interest. The key differentiators that we ‘guess’ to fill in are:
- Persistence – do we track changes? do we compress every N milliseconds/hours/weeks? do we have a limit to how many changes we keep total?
- E-Guard – who can know the name, can read, write, or destroy (permanently-erase) this Info, the conditions/rules by which that can change (purchase? license?). Usually we group all variables into the ‘Aptiv’ Guard for that user but it’s often up to the data’s volatility (again, consider live sensor like a camera or mic or web-feed versus a website password)
- Idea Limits – this handles whether a variable can be resampled to different precision (bit representation for compression covering range) or resolution (can we downsample this variable), if it includes ‘many others’ such as an Image or 3D X-ray Scan as well as propagation, if it’s a ‘Fact’ do we introduce the required delays when reading this value, if it’s owned and possibly updated by others, such as a bank account or score, versus a ‘Belief’ where your local/last version is fine to use and it can be changed retroactively (rollback) if found to have diverged. There is a lot to say on this aspect of ‘Idea Limits’, but the simple answer is that we force the variable into one of 8 types that determines how we handle where it lives, when we sync, what requirements occur to do so, and how it can be adjusted (computationally-clamp, spatially-resample, temporally-resample, use-until-sync’d (belief), sync-before-use (fact), local-operation-only (thought), predictive (chronology), etc. ).
Q6: How is the underlying “stuff” commented so if I lose a “developer” and bring in someone new they understand what they are looking at, and what is the software supposed to be doing?
A6: A key value proposition of wantware is to be able to present ‘what is happening’ as in what variables are loaded, read, written, when and by whom on which chip for which job in ‘your terms’. Initially that’s simple English and over time that’s however you’ve trained it to map the same underlying ‘meanings’ in your terms. E-Powers are unfortunately a Black Box of input-output services, but they do have lots of metadata that can be queried, logged, and put back into sentences.
Every system has its confusions and complexities and I’m NOT claiming our approach is anything but designed to help better present the level of detail you’re seeking to better understand.
Q7: Does the code generated have a design meant to leverage a containerized platform such as Openshift, Docker, and Kubernetes?
A7: If using the Essence-Morpheus scenario, we are able to run in VMs and Containers like any other multi-process application (or single proc on Mobile/MacOS).
Q8: Is there any leveraging of current open source binaries as part of these apps or does this just build everything from scratch?
Q9: From a Maven Repository for example?
A9: So for Apache’s Build Tool Maven, we can add an E-Power that handles Maven Build and all its internal rules directly. Basically, we use the source and wrap it with meanings, statement templates, rules, and claims/proofs (add internally generated unit tests to randomly synthesize input & validate output or skip with a disclaimer/user-opt-in!). We could also directly read and write to Maven Build files in ‘wantware’ starting with ‘open file, read and do this and this, etc.’ That makes it all transparent and everything in Maven (iirc?) calls other tools via shellscript, as we do regularly for building E-Powers, etc.
Q10: If yes how are these scannable to something like Sonotype Nexus software?
A10: For the purposes of Sonotype-Nexus repo use and user distribution, we could write an E-Power that connects to Sonotype’s service and shares both ‘wantware’ as text/meaning-symbols-embedded and E-Powers as binary packages of interface/functionality/validation. However, if you’re using Essence as the runtime, then discovery of other packages is provided P2P, which can come from known associates or commercial entities. Essence ‘Content’ can be embedded on webpages (as binaries), streamed via services (as binary-stream exchange protocols) or directly (machine to machine IP6/IP4 via UDP/ICMP or other protocols).
Q11: How would I reference these libraries?
A11: In the Essence-Chameleon approach, anything you make in wantware is exported into a coding-project, so everything is used as is and if you’re using Essence in an OS (E-Morpheus) or an Appliance(E-Noir), then the idea of libraries only exists as E-Powers (meaning/templates assigned for use, validation-added, metrics taken, etc.) or as cut’n’paste or imported ‘wantware’ (described earlier as plain text with embedded meanings).
Q12: Can the build Artifact utilize unit testing (such as JUnit or JaCoCo I know these are meant for JAVA, I am wondering if any current COTs software can fill this gap)?
A12: Anything that can be run as a shell-command, launched as a Application (with arguments or some initialization-file or registry setup), or handled via network-commands (SSH run?), can be run via wantware mapped into safe, user-friendly/user-designed statement templates for use.
Adding a specific Java Machine (JVM) as an E-Power has worked fine in the past via open source choices and a ‘display window’ for the machine to live within. Providing input/output for a Java runtime in other scenarios (web client) is equally feasible although it carries all the limitations and isolation-requirements that running any emulator does. Wantware can only interface with what we’ve made meanings/verb-templates (think mad-libs) that connect.
Q13: Can I run any code quality scanning against what is being generated?
A13: In the case of E-Chameleon where it’s ‘just code projects’, of course. In terms of the Essence runtimes, there are lots of profiling and runtime scanning tools that apply to the live experience. However, this requires telling your wantware Aptiv to ‘don’t permute this code’, meaning that its generated one time for whatever chips/resources it uses and can be scanned at a fixed location, just like regular code. You can also export your “NO-PERMUTE” code as C99 for Valgrind or SPIR-V for Khronos Validation (LunarG tools), etc. It’s a choice on whether you want self-optimized/regenerated results or fixed-results.
Q14: Security scanning (AppScan) at build time?
A14: Security Scanning, particularly virus-scan-apps, acts the same as the above ‘code-scan’ in terms of options/limits. When we are mapping memory, injecting code live (again, much is linking-snippets/register-rebalancing for CPU/GPU work) and executing/profiling, we tend to be either ignored or heavily red-flagged by security scanners. We do offer limited use of ‘virtual-memory/disk-memory/device-access’ etc. based, as always, on user wants.
Q15: 508 Compliance (DeQue)? Which is going to be a government requirement.
A15: Essence is designed for inclusion and the requirement to make software accessible to more people. Ol is to support as broad an audience as able, from prioritizing text-legibility control, filtering for color-blind-variations, making everything ‘speakable’ (including wantware input and dialog response, so everything can be handled without vision), as well as being able specify rights/controls for any ‘created wantware Asset’, whether the behavior/data described in words or imported other media types.
Q16: Load Testing (LoadRunner, MicroFocus)?
A16: I have not tried this load-testing application but I would expect it works when we run Essence Morpheus and have all the wantware aptivs using that “NO-PERMUTE” mode or perhaps with it? Not sure how their approach works (if it controls our processes and injects stack-profiling code? if it does anything w/ GPU, VideoCodecs, DMA, or USB controllers used for work?). Be fun to try and I’d expect if it can run video-games, web-browsers, or any emulator, it’ll run us fine using ‘FULL-PERMUTE’ wantware Aptivs too.
Q17: Regression Testing (Selenium)?
A17: We could put the Selenium IDE/API into an E-Power and wrap it with meaning/validation to use it inside Essence if desired or you can use it directly with wantware exported as C#/JS/PHP (E-Chameleon) for web browser regression testing. Given its purpose to make certain your webapp runs in the popular browsers, I think we’d have less crossover here than other questions, but it’s certainly useful when either (a) exporting webapps & testing (just use as is!) or (b) when using Essence as a web server to test the client experience.