On Data Languages

March 01 2017

Once upon a time there was SGML. Then HTML happened. Then came XML. After XML rebellion (one israeli, one saxon, a russian and korean), the JSON and YAML were produced.

Then the progress stopped. DL tried to make the next step, but it was single man effort (even though in retrospect - he was the smartest man in the room at that point).

DL failed. Was ahead of times.

I only found: http://www.dougengelbart.org/site/colloquium/forum/ba-ohs-talk/0203/msg00163.html

Then the dark ages of Google started.

Fast forward (almost) 20 years.

What we have at this point: 

HCL looks top of the line at the moment as a language.

The next step is to uplift HCL to YAML / JSON status (implementation in all existing languages)

Could be done in exact way fish is coded: Fish

A week of work.

Another week to uplift UCL to YAML / JSON status, if desired.

At this point it looks very easy to start uplifting little guys into "cross industry standards" (could be fun too).

It's like stealing candy from kids in a way, but that is only because they *are* snowflake government spoiled kids, who sit on all the candy (for no good).

Basically, just need to make a github page, fork the language to uplift - uplift it. Look at other target - repeat.

The trick is to code things right.

As demonstrated by DL, single man efforts don't work well in this domain, so it might help to have two people, not one.

DM me on twitter, if you're interested in participating.

I had to delete twitter account about a year ago, I think. Email me at pault12 gmail, if you want to get involved. For whatever reason. If you can read this page - you rock. (Updated Feb 17 2022)

Update May 30 2019

I've figured out the next step after HCL (HCL was a good rehash of 2-nd wave markup languages, but they are syntax play, not model play). It is amazing how long this stuff takes. But what I have *is* the 3-rd wave. Like hierarchical - relational was.

Update Jan 28 2021

There was a guy in Microsoft in 70s who was trying to figure out the same puzzle. He figured out 90% of it. I am wondering if remaining 10% are worth the hassle, but I think it's foundational, so should be worth it. "Bi-relational" model? Sounds silly, but kind of ok. "N-relational" is not good. "K-relational" is better. Turns out "Bi-relational" is an actual word. So K-relational. Or "II-relational".

Update Feb 10 2022

So it *was* possible to beat the guy from 70s! Remaning 10% were worth the hassle! It is called 'triplets' and it beats all existing data structures! Just wow. I need to experiment a little bit more, but it *is* true post XML move. XML reduced everything to a package of scalar / hash / list. I understand why they did it, because it's the basics of AST, very hard to *not* do it that way, but if you change the angle a bit - you get triplets. In a way it's similiar to nginx / apache dilemma (nginx is not a server, it's a network driver on steroids). Both can be called "servers" but that's semantics. Two different instruments. Triplets are to XML what nginx is to apache. Took 20 years to get them. Let's see if they hold. So far - I see no holes.

Update Feb 17 2022

So far so good. Triplets clearly work for data language, smooth sailing. Interestingly enough, first (and only) assembly languages were also triplets. There is an important difference. Of course 3 slots is not much. So they had to decide what to drop. Very hard to fit the naming into so few spots, so they dropped the naming. That's because machines don't need no names. Humans do need names. Obviously. That's how Macro assembly emerged (and then C inherited from that). The only purpose of Macro assembly was to add naming layer on top of nameless operations. Now what do we have here. The pattern is the same, but my machine is not pure hardware, so names want first seat. Let's see how it goes.

Update Feb 19 2022

It works. It moved from one application to another and the foundations hold. It's effectivy a hybrid between macro assembly and .ini files. It's kind of weird, of course, but then again because it's a macro assembly that has no limitations imposed by hardware, of course it is a bit weird. Being able to express second application through it means it scales and that's the only thing that matters. I am facing too much of a hard cases on every step, so it will take this thing a long time to evolve. It took 20+ years to get it. It's 50 years if we count from .ini files (UNIX was 1971 and .ini files were a (questionable) transformation on UNIX configs).

Another project of this kind is this: Pfs - if you think the name sounds familiar most likely that's because you heard of iPfs. Which emerged about the same time I did my little project. This website of mine is full of this kind of coincidences. Silicon valley is a place where some people are gods and some are bums. Most are of course bums. Gods can't stay here for long. Usually they move to Mars or something. Gods are very busy. Always on the run.

Update Feb 26 2022

It definitely works. Everything in UNIX is a file, everything in a file is a field, so plenty of things can be expressed with triplets ( + conditions ). Surprisingly, the closest to this was Forth. The vocabulary grows on it's own organically (simply by adsorbing a very minimal subset of UNIX text toolig in most primitive way), and now there is a way to have logical pipelines of sorts, only instead of | the data is pulled into triplet and get mixed and mashed with conditions. To replace triplets one will have to write awk scripts, but that's the whole point. Triplets is *data* language, not the scripting language. And it works. 

tp:500
sl:-500
pfx:BTCpigBmyre
pause:0
eod:15
abid:/code/BTC.log:awk 3
delta:300
bos:BTC.BTCmyre.ST:awk 1
lvl:BTC.BTCmyre.ST:awk 2
bcond:( ("$bos" eq "B") && ($abid < ($lvl-$delta)) ):eval
scond:0:eval

Update Mar 22 2022

Everything works fantastic. The closest is Forth.

Update Mar 26 2022

It's weird blend of Forth and XT. Processing model of XT/Forth blended with vocabularies XML/Forth. All parts simplified and generalized. This means XML was ahead of it's times (not surprisingly). This also spells some doom to some very expensive modern technologies. Pfs did the same. 

dthms::date
ll://this.is.snap/vav-snap.php:wget
l:ll:split
# bidb bid rsib rsi momb mom pb pp
# 1    2   3    4   5    6   7  8
erb:l3:en -40 -20 -10 0 10 20 40 
emb:l5:en -8 -4 0 4 8
epb:l7:en -8 -4 -2 0 2 4 8
out:$dthms $l1 $erb$emb$epb:fmt
Surpisingly, this is more maintainable, comparing to original code.

Update Apr 09 2022

XML - mixed content (- namespaces of course) = YAML (python ws trap) | JSON (suboptimal because no %) | S-exprs+ (because yes %/@/$)
S-exprs+ + X = what plants crave. Figured out X{}. This merges 2 pages. DSL and DL.

Update Apr 14 2022

---
sub DL::rdol($){ my $a = shift; my $ret = ($a eq "L") ? "U" : "L"; return $ret; }
---
dthms::date
bid:/code/HANJ/hanj.EURUSD.1.log:awk 3
inEU:root@10.0.0.105 /tmp/Sbond.csv:rawk 4
sbEU:root@10.0.0.105 /tmp/inEUR.csv:rawk 3
einEU:$inEU>0:eval
esbEU:$sbEU>0:eval
aDOLL:root@10.0.0.106 /u2/Acha/FEEDS/DOLLAR:rawk 6
rDOLL:rdol("$aDOLL"):eval
aEU:root@10.0.0.106 /u2/Acha/FEEDS/EURUSD:rawk 6
aGU:root@10.0.0.106 /u2/Acha/FEEDS/GBPUSD:rawk 6
aNAS:root@10.0.0.106 /u2/Acha/FEEDS/NAS100:rawk 6
aBTC:root@10.0.0.106 /u2/Acha/FEEDS/BTCUSD:rawk 6
#out:$dthms $bid $inEU $sbEU $aDOLL $aEU $aGU $aNAS $aBTC $einEU $esbEU:fmt
out:$dthms $bid $rDOLL$aEU$aGU$aNAS$aBTC$einEU$esbEU:fmt

Update Apr 15 2022

Good bye, Relational Model. You had a good run. Since 1969. Thank you for creating plenty of fortunes and drama on this planet. As a matter of fact, your first challenge emerged in 1989 (20 years after you were born). By 2009 you should have been gone, really, but the planet got into this "restructuring". You got lucky you were able to hang around a bit longer than you should have. I wish you a nice retirement in Hawai. Now I am looking at you, UCSD Pascal. You are from 1977. You're next, dude.

Update Apr 17 2022

f:STL5AFBCH
fl:STL5AFBCH.log
fd:/code/BFEED_PLAIN/
feed:$fd$fl:fmt
ttls:BTC stl btc5afint bchv
cols:black blue green red
mult:0.005 1000 0.05
pfx:bcha
outd:/u2/Bcha/IMG

This makes it clear. If static - can live with tuples. The moment you want dynamism - *have* to use triplets. I think that's what they wanted to accomplish with EJB. Well, they failed. They failed 20 years ago

Update Dec 05 2022

So this RDB stuff was most certainly a joke. Knowing K==3 ask the rest of the world to handle K==2. Very much like crypto bubble. The joke worked since 70s, but it no longer can. That makes noSQL a joke on top of a joke. Not bad. Hadoop is Bigus Datus (useless at that). I seen the K==3 implemented, only it was in 2000s so I ignored it's existence. Better late than never, I guess.

Update Feb 05 2023

The Z80 also works from the simplified world view that we do not use 3-operand instructions. - they failed not 20 years ago but 50 years ago. It's not that they "failed" per se. Group is split in two and one part of the group has to implement bad design and another part is given a winning design and then they "fight". Sounds familiar? Because it is familiar. The valley was ahead of the planet by 50 years. This also explains the (fake) fight between Dutch and Deutsche.

Update May 20 2023

The list for YAML now reads as: Ingy, Tina, Panto, Eemeli and Thom
It does not mention Oren or Clark even. Of course it will never mention Don and Paul.
And that is ok. Internetz is for outcasts. Also - I delivered on UCSD Pascal. Now need to understand what to do with all this in the middle of South Africa.

Update May 23 2023

So the S-expressions... Every time anybody will try to actually solve the DL Acorn puzzle, they will be hit with :

If the goal is to have a language with an unambiguous input syntax that is capable of making type inferences from the input stream, then what is wrong with the LISP syntax? It does exactly the job you're doing, has names and namespaces, and is incredibly easy to parse. The goal of any schema language is to *constrain* the structure of a particular input stream. If we could marry the LISP syntax to a decent schema language (and allow the schema to be optional), we'd have the best of both worlds.

S-expressions is a trap. Who cares they are easy to parse? They are actually *not* easy to parse. Have you tried actually using S-expressions parser? I did. I got away with macros, of course. Who knows how to use macros properly? *Nobody*. Should have written a few compilers for that kind of skill. Do you know many people, who happen to write several compilers? Yeah. You don't know. They don't exist. Except for me and a few other guys, whom I know (and you don't know). Nevermind. I am reading various EU archives, committees are big in EU. Going strong for hundreds of years, huh.

Update Jun 05 2023

Astonishing update. The dude, who was behind DL, had fallen into S-expressions trap! Bradford And he is in Switzerland (naturally). Ha! Well, dude, you're basically 30 years behind now. Well, not only you, of course. Boston is 30 years behind on *everything*. Wow. I was *not* expecting this. Valley is clearly sponsoring Switzerland now. I wonder how long that can possibly last.

Update Sep 17 2023

Hierarchical / Relational is effectivly Bayes / Markov.

Update Oct 19 2023

Not every day one invents the fundamentally new storage engine. Yeah. ( Time/Distance predicate on steroids. They do plenty of that in DAG, but *not* on time. Ironically, I first moved into that direction when interviewed by one dude here, many years ago, I of course did not get a job, because dude was smart enough to sense that I am inventing some unusual optimization algo realtime in front of his eyes. Why would anybody hire somebody 10x better than them on some aspect? Not many people do.)

Update Oct 21 2023

So why *exactly* there is not a single engine out there blending relational/hierarchical/network ? Evolution (degradation) of CSS gives a good idea why. One day I might restore bixpath. It was 10+ years ahead of everything.



Oct 28 2024.

Major breakthrough on semiotics, DLs and DSLs. Consolidating into AI page