"Download Blockchain" ButtonsteemCreated with Sketch.

in #steemdev6 years ago


new steemit tech.jpg

Have you heard?! There's a new crypto in town that let's you earn rewards just by posting your thoughts to the blockchain! It's called Steem and I hear it uses something called distributed ledger technology, whatever that means! Apparently, hundreds of servers all around the world are storing the same information in consensus so we know it can be trusted and hasn't been tampered with. Anyone can download the ledger and verify it for themselves! Amazing!

Cool, so where is it?

Huh?

How do I download the ledger?
I don't see it anywhere.

... ... ... ... ...



question.png

Hm yeah, neither do I. Where is our ledger? Why is it not being prominently displayed? We have the audacity to boast how decentralized and trustworthy we are without presenting the evidence upfront? Seems flimsy, at best. Last I heard, the blockchain was 150 gigs of raw data. Well, guess what? I have 150 gigs free space, along with many others, and I want to download the blocks. Where is the download blockchain button?

Certainly one reason for the overstep must be due to the process of actually validating the information. It might 'only' take 150 gigs to download all the raw data, but confirming that data is accurate is another story. A lot of the information needs to get pumped straight into RAM, which is a lot more expensive and rare than hard drive space. Steemit has switched gears and is attempting to rectify this problem right now with the switch to RocksDB.

A part of me has to ask, "How was this not a priority until now?" If the bear market was the catalyst for this change of focus, what was Steemit up to when coins were only worth 10 cents? However, I can accept that mistakes were made and move on. That's fine. What isn't fine is the complete inability to easily share information in a decentralized manner.

If I want to download those blocks, who is going to pay for it? Is a Steemit node going to upload all 30 million of them directly to my computer? What if 10,000 other users want to download those blocks at the same time? This kind of centralized distribution doesn't scale.


micro torrent.png

The embarrassing thing about all of this is that the technology to disseminate information in a decentralized fashion was invented 18 years ago in 2001 by Bram Cohen. BitTorrent is almost twice as old as Bitcoin itself, but somehow only Justin Sun and Tron have figured out that these two technologies compliment each other in a big way. Where is the Bitcoin torrent? Where is the Ethereum torrent? Where is the Steem torrent?

This question makes a lot more sense in the context of Steem. In comparison, the information on our blockchain is much more interesting and useful than what can be found on the Bitcoin blockchain. Again I ask: where are these torrents?

The ability to hash a file to verify it hasn't been tampered with has also existed for a very long time. Why are Steem nodes forced to sync data that they know for a fact has already been verified, even by their own node in the past? We are working a lot harder than we are smartly.

What about second layer assets that exist on top of the Steem blockchain? There are a lot of applications out there that make a lot more sense if they simply trust the information on the blocks instead of having to verify it themselves. Using this trust layer, we can develop second layer scaling solutions that will work just fine as long as the block producers stay honest. It is what we pay them for, afterall.


matchmaking lobby.png

For example, let's say someone enters a matchmaking lobby to find an opponent to play against. Let's assume there are no centralized nodes to process these requests; each client must act as its own node. This would not be possible if they were forced to validate the blocks they received, but it is trivially easy if all they have to do is download the last 100 blocks. They assume those blocks are legit and scan them to see if anyone else is also looking to play the game. If they can't find anyone, they post to the blockchain that they are looking and wait for someone else to come along and scan their request; a strange game of leap frog is born.

None of this is possible unless blocks are distributed by torrent streams. We can't employ any kind of centralized solution for these problems or they will bottleneck and fail before they can scale to any kind of meaningful level.

I know for a fact that many members of this community would seed blockchain torrents for free. I know I would. 24/7. I don't need a reward to make this platform stronger. My reward is the stake I own increases in value along with everyone else. This is the correct way to think and interact with a decentralized community,

The needs of the many...

Sort:  

Have you heard of ZeroNet?

Posted using Partiko Android

I have now! :D

ZeroNet is a decentralized computer network of peer-to-peer users built in Python and is a fully open source project. It uses BitTorrent and Bitcoin Cryptography to make it decentralized and impervious to censorship network.

It works in a similar way just like you share torrent files on the BitTorrent P2P network. The difference is that here a website is shared instead of pirated movies and TV shows.A website uploaded on ZeroNet isn’t hosted on a central server but by the users who access it, just like the torrent network in which the data is pulled from the machines which are downloading the torrent.

Now, where is the "tokanize this app" button :).

Posted using Partiko Android

Good, you'll have a ball there. I have to resurrect my account for the email I had through them.

Posted using Partiko Android

Hey I started reading about a crypto called Holo. It looks like it operates almore identically to the way Zeronet does

Thanks for the heads up I just finished reading about it and it seems very promising indeed especially since it rewards work/resources directly peer to peer, no more POS.

Posted using Partiko Android

Distributing the block chain backlog and running full consensus nodes is a problem in any most blockchains. I'd love to see consensus node requirements continue to fall. I'd be happy seeding distribution too ... Torrent or whatever. Maybe even IPFS.

Posted using Partiko Android

Great thinking.
I can totally understand you because Ive gone through the same thoughts.

In summary there is still really much potential!
Many technologies can collaborate or do some symbiosis.. ;)

We will see if steem realises them, or not.. ^^

Posted using Partiko Android

Gotta admit, I wonder about both of those as well. Bittorrent and hashing seem way overdue, and no brainers in terms of reducing server loads.

Thanks!

You are trying to solve a problem that is not there or already solved.
I guess you also have to re-think that "downloading" the steem DB (or the BTC blocks) pre-verified via a torrent exactly is NOT decentralization.

Many blockchains offer the a pre-verified blocks db snapshot via https/torrent/rsync/ftp for convenience, but if you did not sync the blocks yourself, you have to trust the source of this snapshot to give you a geniune one.
On the other hand, syncing from multiple peers starting at genesis, you can be sure to have a clean and trustworthy blocks database, because your own node applied consensus verification on all blocks itself.
Syncing/requesting blocks from multiple peers is fast and decentralized, most blockchains implement a DHT for that. So "syncing" is like torrent - but better.

It's a bit arrogant to assume the problem is solved when 99% of the people here have no idea how to do it or what the requirements are.

Quick! I want to download the most recent 10,000 blocks and tinker with them using a database. Doesn't even matter if the blocks are bad (which they aren't). How does one do that without being a programmer? Impossible right now.

Even your lack of commitment in saying

a problem that is not there or already solved.

is extremely telling. Which is it? It should be obvious.

if you did not sync the blocks yourself, you have to trust the source of this snapshot to give you a geniune one.

So what happens when every witness and every node verifies the snapshot? Can we trust it then?

What happens when we don't really care if we can trust the snapshot? There are plenty of situations where entire applications could be developed using invalid blocks.

Trust doesn't always matter. In fact, the foundation of DPOS, and many other scaling solutions like Sharding, involves sacrificing trust for efficiency. I'm just pointing out we need to bring that concept to the next level.

If you want the last 10000 blocks... ask a node!
The node will give them to you. Of course you need to speak to the node in its language, its API.

If you want the whole steem blockchain as a regular DB, follow this guy's howto and use his snapshot.
https://steemit.com/hivemind/@emrebeyler/steps-to-restore-a-hivemind-database-snapshot

Call me arrogant if you like, but

How does one do that without being a programmer? Impossible right now.
if you are not a programmer, why on earth would you like to download blocks?
If you are not a programmer, use a shiny frontend trusting its vendor ;-)
Or if you can at least do some basic coding, use the many frameworks like beem to interact with the blockchain.

STEEM doesn't have thin client wallets yet with two party authentication, so i'm going to wait until it matures a little bit.

While I fully agree with you with respect of this view for the long term as adoption increases and must ask for pardon if it is a dumb question due to my limited tech savvyness, is not this the job of the witnesses? They hold and run the nodes to ensure its distributed nature? While there are only the top 20 doing the majority, I often here some above and in the top 100 also producing blocks for their nodes. Being as vested and seeing the long term potential, I would love to run a node myself ultimately but agree that it should be in a more distributed and efficient method.

Posted using Partiko iOS

The job of the witnesses is to produce good blocks. Running a full node like Steemit does is expensive and many witnesses aren't doing it. If someone is developing an app and wants to ask the database a question they have to query a full node using the Steemit Application Protocol Interface. There aren't a lot of full nodes to choose from and there are a lot of flaws in the API.

For example, if you ask the API for a discussion (blog/comment post) it will send you back the correct Discussion object. What does a discussion object contain? Surely, it must contain the block number that it was created and modified on. NOPE! And to me, this is blatant incompetence, meaning if I want to put any real work in I might as well just download all the blocks myself and database/interpret them myself in a not so idiotic way.

https://steemit.com/incompetence/@edicted/more-steemit-inc-incompetence

https://steemit.com/utopian-io/@edicted/fixing-steem-with-baby-steps-part-10-database-api-necessities

My perspective is probably hard to interpret because no one is trying to do what I want to accomplish. I want to program real dapps; ones that do not rely on centralized nodes to make them work. Essentially I want every client to also be a server. I want everyone who uses my dapp to be their own node.

And so, if I ended up making a dapp that millions of people are trying to use, that means millions of these light nodes are going to need to be downloading blocks. At a certain point, the centralized way we are distributing blocks is going to fail spectacularly. Which means these light nodes will be forced to share blocks with each other instead of relying on the full nodes.

At this point we have a situation where light nodes could be feeding incorrect information to other light nodes. I'm just trying to jump in front of this problem and make the claim that everyone should just be sharing blocks with everyone and there should be an easy way to verify that we all got the correct information.

Congratulations! This post has been upvoted from the communal account, @minnowsupport, by Edicted from the Minnow Support Project. It's a witness project run by aggroed, ausbitbank, teamsteem, someguy123, neoxian, followbtcnews, and netuoso. The goal is to help Steemit grow by supporting Minnows. Please find us at the Peace, Abundance, and Liberty Network (PALnet) Discord Channel. It's a completely public and open space to all members of the Steemit community who voluntarily choose to be there.

If you would like to delegate to the Minnow Support Project you can do so by clicking on the following links: 50SP, 100SP, 250SP, 500SP, 1000SP, 5000SP.
Be sure to leave at least 50SP undelegated on your account.