On Data Languages

March 01 2017

Once upon a time there was SGML. Then HTML happened. Then came XML. After XML rebellion (one israeli, one saxon, a russian and korean), the JSON and YAML were produced.

Then the progress stopped. DL tried to make the next step, but it was single man effort (even though in retrospect - he was the smartest man in the room at that point).

DL failed. Was ahead of times.

I only found: http://www.dougengelbart.org/site/colloquium/forum/ba-ohs-talk/0203/msg00163.html

Then the dark ages of Google started.

Fast forward (almost) 20 years.

What we have at this point:

HCL looks top of the line at the moment as a language.

The next step is to uplift HCL to YAML / JSON status (implementation in all existing languages)

Could be done in exact way fish is coded: Fish

A week of work.

Another week to uplift UCL to YAML / JSON status, if desired.

At this point it looks very easy to start uplifting little guys into "cross industry standards" (could be fun too).

It's like stealing candy from kids in a way, but that is only because they *are* snowflake government spoiled kids, who sit on all the candy (for no good).

Basically, just need to make a github page, fork the language to uplift - uplift it. Look at other target - repeat.

The trick is to code things right.

As demonstrated by DL, single man efforts don't work well in this domain, so it might help to have two people, not one.

DM me on twitter, if you're interested in participating.

I had to delete twitter account about a year ago, I think. You think dealing with nazis will be walk in a park? Of course not. Email me at pault12 gmail, if you want to get involved. For whatever reason. If you can read this page - you rock. (Updated Feb 17 2022)

Update May 30 2019

I've figured out the next step after HCL (HCL was a good rehash of 2-nd wave markup languages, but they are syntax play, not model play). It is amazing how long this stuff takes. But what I have *is* the 3-rd wave. Like hierarchical - relational was.

Update Jan 28 2021

There was a guy in Microsoft in 70s who was trying to figure out the same puzzle. He figured out 90% of it. I am wondering if remaining 10% are worth the hassle, but I think it's foundational, so should be worth it. "Bi-relational" model? Sounds silly, but kind of ok. "N-relational" is not good. "K-relational" is better. Turns out "Bi-relational" is an actual word. So K-relational. Or "II-relational".

Update Feb 10 2022

So it *was* possible to beat the guy from 70s! Remaning 10% were worth the hassle! It is called 'triplets' and it beats all existing data structures! Just wow. I need to experiment a little bit more, but it *is* true post XML move. XML reduced everything to a package of scalar / hash / list. I understand why they did it, because it's the basics of AST, very hard to *not* do it that way, but if you change the angle a bit - you get triplets. In a way it's similiar to nginx / apache dilemma (nginx is not a server, it's a network driver on steroids). Both can be called "servers" but that's semantics. Two different instruments. Triplets are to XML what nginx is to apache. Took 20 years to get them. Let's see if they hold. So far - I see no holes.

Update Feb 17 2022

So far so good. Triplets clearly work for data language, smooth sailing. Interestingly enough, first (and only) assembly languages were also triplets. There is an important difference. Of course 3 slots is not much. So they had to decide what to drop. Very hard to fit the naming into so few spots, so they dropped the naming. That's because machines don't need no names. Humans do need names. Obviously. That's how Macro assembly emerged (and then C inherited from that). The only purpose of Macro assembly was to add naming layer on top of nameless operations. Now what do we have here. The pattern is the same, but my machine is not pure hardware, so names want first seat. Let's see how it goes.

Update Feb 19 2022

It works. It moved from one application to another and the foundations hold. It's effectivy a hybrid between macro assembly and .ini files. It's kind of weird, of course, but then again because it's a macro assembly that has no limitations imposed by hardware, of course it is a bit weird. Being able to express second application through it means it scales and that's the only thing that matters. I am facing too much of a hard cases on every step, so it will take this thing a long time to evolve. It took 20+ years to get it. It's 50 years if we count from .ini files (UNIX was 1971 and .ini files were a (questionable) transformation on UNIX configs).

Another project of this kind is this: Pfs - if you think the name sounds familiar most likely that's because you heard of iPfs. Which emerged about the same time I did my little project. This website of mine is full of this kind of coincidences. Silicon valley is a place where some people are gods and some are bums. Most are of course bums. Gods can't stay here for long. Usually they move to Mars or something. Gods are very busy. Always on the run.

Update Feb 26 2022

It definitely works. Everything in UNIX is a file, everything in a file is a field, so plenty of things can be expressed with triplets ( + conditions ). Surprisingly, the closest to this was Forth. The vocabulary grows on it's own organically (simply by adsorbing a very minimal subset of UNIX text toolig in most primitive way), and now there is a way to have logical pipelines of sorts, only instead of | the data is pulled into triplet and get mixed and mashed with conditions. To replace triplets one will have to write awk scripts, but that's the whole point. Triplets is *data* language, not the scripting language. And it works.

tp:500
sl:-500
pfx:BTCpigBmyre
pause:0
eod:15
abid:/code/BTC.log:awk 3
delta:300
bos:BTC.BTCmyre.ST:awk 1
lvl:BTC.BTCmyre.ST:awk 2
bcond:( ("$bos" eq "B") && ($abid < ($lvl-$delta)) ):eval
scond:0:eval

Update Mar 22 2022

Everything works fantastic. The closest is Forth.

Update Mar 26 2022

It's weird blend of Forth and XT. Processing model of XT/Forth blended with vocabularies XML/Forth. All parts simplified and generalized. This means XML was ahead of it's times (not surprisingly). This also spells some doom to some very expensive modern technologies. Pfs did the same.

dthms::date
ll://this.is.snap/vav-snap.php:wget
l:ll:split
# bidb bid rsib rsi momb mom pb pp
# 1    2   3    4   5    6   7  8
erb:l3:en -40 -20 -10 0 10 20 40 
emb:l5:en -8 -4 0 4 8
epb:l7:en -8 -4 -2 0 2 4 8
out:$dthms $l1 $erb$emb$epb:fmt
Surpisingly, this is more maintainable, comparing to original code.

Update Apr 09 2022

XML - mixed content (- namespaces of course) = YAML (python ws trap) | JSON (suboptimal because no %) | S-exprs+ (because yes %/@/$)
S-exprs+ + X = what plants crave. Figured out X{}. This merges 2 pages. DSL and DL.

Update Apr 14 2022

---
sub DL::rdol($){ my $a = shift; my $ret = ($a eq "L") ? "U" : "L"; return $ret; }
---
dthms::date
bid:/code/HANJ/hanj.EURUSD.1.log:awk 3
inEU:root@10.0.0.105 /tmp/Sbond.csv:rawk 4
sbEU:root@10.0.0.105 /tmp/inEUR.csv:rawk 3
einEU:$inEU>0:eval
esbEU:$sbEU>0:eval
aDOLL:root@10.0.0.106 /u2/Acha/FEEDS/DOLLAR:rawk 6
rDOLL:rdol("$aDOLL"):eval
aEU:root@10.0.0.106 /u2/Acha/FEEDS/EURUSD:rawk 6
aGU:root@10.0.0.106 /u2/Acha/FEEDS/GBPUSD:rawk 6
aNAS:root@10.0.0.106 /u2/Acha/FEEDS/NAS100:rawk 6
aBTC:root@10.0.0.106 /u2/Acha/FEEDS/BTCUSD:rawk 6
#out:$dthms $bid $inEU $sbEU $aDOLL $aEU $aGU $aNAS $aBTC $einEU $esbEU:fmt
out:$dthms $bid $rDOLL$aEU$aGU$aNAS$aBTC$einEU$esbEU:fmt

Update Apr 15 2022

Good bye, Relational Model. You had a good run. Since 1969. Thank you for creating plenty of fortunes and drama on this planet. As a matter of fact, your first challenge emerged in 1989 (20 years after you were born). By 2009 you should have been gone, really, but the planet got into this "restructuring". You got lucky you were able to hang around a bit longer than you should have. I wish you a nice retirement in Hawai. Now I am looking at you, UCSD Pascal. You are from 1977. You're next, dude.

Update Apr 17 2022

f:STL5AFBCH
fl:STL5AFBCH.log
fd:/code/BFEED_PLAIN/
feed:$fd$fl:fmt
ttls:BTC stl btc5afint bchv
cols:black blue green red
mult:0.005 1000 0.05
pfx:bcha
outd:/u2/Bcha/IMG

This makes it clear. If static - can live with tuples. The moment you want dynamism - *have* to use triplets. I think that's what they wanted to accomplish with EJB. Well, they failed. They failed 20 years ago