improve documentation files (#218)

* add all the docs

* fix linebreaks

* fix a few links

* note on dev branch

* spelling

* fix tabs

* address PR comments by Tyler

* fix docs readme markup

* fix inlining on the whole directory-tree

* prettify docker instructions

add a whale, etc.

* reorganize /docs/readme.md sections

* use table for dependency references

* refactor and reference whitepapers

* move process and guidelines to wiki

* refactor readme front

* add link to Orion to root readme as well

* use plain links

* not on scala

* add fancy discord linker to readme

* less lines by removing db from tree

* add more introductory text

* validate all readme frontpage links by linking to dev branch

* add youtube video that now has slides

* deploy and terraform readme cleanups

* fix diagram inclusions

* redirect links from feature to dev branch

README.md

@@ -1,11 +1,17 @@
 ## Constellation
-Decentralized Application Integration Platform.
-
-This repository is the reference implementation of our 
+This repository is the reference implementation of our **Decentralized Application Integration Platform** using the 
+[Scala](https://www.scala-lang.org/)
+programming language. We build a horizontally scalable 
 [DAG](https://en.wikipedia.org/wiki/Directed_acyclic_graph) 
-protocol using the 
-[Scala](https://www.scala-lang.org/) 
-programming language.
+protocol that you can build upon and interface with using common software standards.
+
+### :books: Resources
+Visit the repository
+[**wiki**](https://github.com/Constellation-Labs/constellation/wiki) 
+for developer tools and documentation and consult the 
+[**/docs**](https://github.com/Constellation-Labs/constellation/tree/dev/docs).
+for an explainer of the protocol archetecture or an extended overview of the 
+[project structure](https://github.com/Constellation-Labs/constellation/blob/dev/docs/directory-tree.md).
 
 ```scala
 src
@@ -21,31 +27,26 @@ src
     └── util
           API.scala
           ConstellationNode.scala
-          LevelDB.scala
 └── test/scala/org/constellation          <─── unit tests
 ```
 
-## Development
-### :books: Resources
-For an explainer of the protocol archetecture or an extended overview of the 
-[project structure](https://github.com/Constellation-Labs/constellation/docs/directory-tree/), 
-visit the 
-[**/constellation/docs**](https://github.com/Constellation-Labs/constellation/docs/).
-Consult the
-[**repository wiki**](https://github.com/Constellation-Labs/constellation/wiki) 
-for developer tools and documentation. 
-
 ### :computer: Build instructions
-For details on the build process, as well as pointers for docker, vagrant and deployment, see [/docs/build-instructions](https://github.com/Constellation-Labs/constellation/blob/developer/nikolaj/add-docs/docs/build-instructions.md).
+For details on the build process, as well as pointers for docker, vagrant and deployment, see [/docs/build-instructions](https://github.com/Constellation-Labs/constellation/blob/dev/docs/build-instructions.md).
 
-### :two_hearts: Community
-For questions and contributions, can find links to the community outlets and more in the 
-[**resource list**](https://github.com/Constellation-Labs/awesome-constellation). Our knowledge, news and discussion outlet is the [**Orion**](https://orion.constellationlabs.io/) discourse forum. Join our [Discord](https://discordapp.com/) server for a chat. 
+### :green_book: API Docs
+We intend to use **Swagger** to publish comprehensive API docs.
 
 ### :rotating_light: Troubleshooting
-For issues and bug reports, see [/wiki/Contribution-guidelines](https://github.com/Constellation-Labs/constellation/wiki/Contribution-guidelines). 
-There you also find general pointers toward the development process. 
-Note that the `master` branch might be behind the `dev` branch by a few weeks.
+For issues and bug reports, see [wiki/Contribution-guidelines](https://github.com/Constellation-Labs/constellation/wiki/Contribution-guidelines). 
+There you also find general pointers toward the development process. Note that the `master` branch might be behind the `dev` branch by a few weeks.
 
-### :green_book: API Docs
-We intend to use **Swagger** to publish comprehensive API docs.
+### :two_hearts: Community
+For questions and contributions, can find links to the community outlets and more in the 
+[**resource list**](https://github.com/Constellation-Labs/awesome-constellation). 
+Our knowledge- and news-outlet is the 
+[**Orion**](https://orion.constellationlabs.io/) 
+Discourse forum. To quickly reach out, join our [Discord](https://discordapp.com/invite/KMSmXbV) server for a chat:
+
+  <a href="https://discordapp.com/invite/KMSmXbV">
+	  <img src="https://img.shields.io/badge/chat-discord-brightgreen.svg"/>
+  </a>

deploy/README.md

@@ -1,18 +1,20 @@
 Ensure
 
+```
+Host * 
+StrictHostKeyChecking no
+```
 
-Host *
-    StrictHostKeyChecking no
+Present in `~/.ssh/config` before running pssh commands.
 
-Present in ~/.ssh/config before running pssh commands
-
-
-Using GCP Compute instances temporarily instead of Kubernetes due to low entropy bug
+We're using GCP Compute instances temporarily instead of Kubernetes due to low entropy bug.
 
 Workflow:
 
-create-cluster-from-snapshot.sh <num-machines> <node-label> for bringing up machines
+`create-cluster-from-snapshot.sh <num-machines> <node-label>`
+
+for bringing up machines
 
-update-hosts.sh <hosts-file> <node-label>
+`update-hosts.sh <hosts-file> <node-label>`
 
-redeploy.sh <hosts-file> <node-label>
+`redeploy.sh <hosts-file> <node-label>`

docs/architecture.md

@@ -1,11 +1,22 @@
-### Architecture documentation
+## Architecture documentation
+### Diagrams
+#### ledger architecture
+![
+from /docs/images/architecture-diagrams/ledger-architecture.png
+](
+https://github.com/Constellation-Labs/constellation/blob/dev/docs/images/architecture-diagrams/ledger-architecture.png
+)
 
-https://github.com/Constellation-Labs/constellation/blob/developer/nikolaj/add-docs/docs/images/architecture-diagrams/ledger-architecture.png
+#### node-architecture
+![
+from /docs/images/architecture-diagrams/node-architecture.png
+](
+https://github.com/Constellation-Labs/constellation/blob/dev/docs/images/architecture-diagrams/node-architecture.png
+)
 
-https://github.com/Constellation-Labs/constellation/blob/developer/nikolaj/add-docs/docs/images/architecture-diagrams/node-architecture.png
-
-https://github.com/Constellation-Labs/constellation/blob/developer/nikolaj/add-docs/docs/images/architecture-diagrams/protocol-data-flow-sketch.png
-
-(todo: turn all images to proper links with alt text and add explainer for all the diagrams)
-
-(trial: [[link-to-image.png|alt=protocol-data-flow-sketch]])
+#### protocol-data-flow-sketch
+![
+from /docs/images/architecture-diagrams/protocol-data-flow-sketch.png
+](
+https://github.com/Constellation-Labs/constellation/blob/dev/docs/images/architecture-diagrams/protocol-data-flow-sketch.png
+)

docs/build-instructions.md

@@ -2,19 +2,19 @@
 ### :computer: On Linux and Mac
 1. Check out the repository
 
-```haskell
+```bash
 git clone [email protected]:Constellation-Labs/constellation.git
 ```
 
 2. From root directory `constellation`, run 
 
-```haskell
+```bash
 ./build.sh
 ```
 
 or optionally (to connect to other host)
 
-```haskell
+```bash
 ./build.sh seedhost:port
 ```
 
@@ -33,13 +33,13 @@ Note: *For now this installation is not covered here in detail.*
 2. Follow directions for building for development.
 3. Run
 
-```haskell
+```bash
 sbt docker:publishLocal
 ```
 
 4. Run
 
-```haskell
+```bash
 ./run-local-docker.sh
 ```
 
@@ -52,7 +52,7 @@ This is **deprecated** but may be useful for people running Windows, etc.:
 
 1. Download [vagrant](https://www.vagrantup.com).
 2. Run 
-```haskell
+```bash
 vagrant up
 ```
 from project directory. See also

docs/design-choices.md

@@ -1,28 +1,34 @@
 ### On design choices
 
 This file summarizes current resoning for our approach and design decissions, 
-reviews and references. Please communicate suggestions by making a thread on the 
-[community portal Orion](https://orion.constellationlabs.io/accounts/login/?next=/) 
-or approaching the developers on the
-[discord](https://discordapp.com/invite/KMSmXbV)
-server.
+reviews and references. 
 
 Many decissions can be traced back to the fundamental goal to provide and
 accessible, scalable protocol that focuses on solving the consensus task. 
 For a cursory glipse into the core teams perspectives, you may check out the following clips:
 
-* [youtube.com/constellation-labs/talk-at-Tech-Crunch](https://youtu.be/SsYZF4msXuQ) (Aug. 2018, 40 mins)
-* [youtube.com/constellation-labs/testnet-overview](https://youtu.be/xjn6Te7Twg4) (Sept. 2018, 22 mins)
-
-#### on architecture
+* [youtube.com/constellation-labs/talk-at-Tech-Crunch](https://youtu.be/fCscJL3_tdU) (Oct. 2018, 28 mins)
+* [youtube.com/constellation-labs/testnet-overview](https://youtu.be/SsYZF4msXuQ) (Aug. 2018, 22 mins)
+
+#### Why scala?
+There are some general notes on Scala and also on other functional programming languages actively used for crypto projects in the
+[/wiki/Comparisons-to-other-protocols](https://github.com/Constellation-Labs/constellation/wiki/Comparisons-to-other-protocols#fast_forward-projects-using-a-functional-language-approach).
 
-(tbd)
+One motivating factor as language of choice for the reference implementation of the protocol was of course the core teams experience with it, as well as the useful packages like akka actors and apache spark on the Java virtual machine (JVM). The constellation code base also makes extensive use of the type hierarchy features. In fact, the para-protocol approach to dApp integration builds on it.
 
-#### on recommended-frameworks
+#### On architecture
 
-(tbd)
+For diagrams, see 
+[/docs/architecture.md](https://github.com/Constellation-Labs/constellation/blob/dev/docs/architecture.md).
 
-#### on dependencies
+#### Feedback
 
-(tbd)
+Please communicate suggestions by making a thread on the 
+[community portal Orion](https://orion.constellationlabs.io/accounts/login/?next=/) 
+or approaching the developers on the 
+[discord](https://discordapp.com/invite/KMSmXbV) 
+server:
 
+  <a href="https://discordapp.com/invite/KMSmXbV">
+	  <img src="https://img.shields.io/badge/chat-discord-brightgreen.svg"/>
+  </a>

docs/directory-tree.md

@@ -100,7 +100,5 @@ root
         	UtilityTest.scala
 └── ui
 ```
-	 
-(todo: add remaining files)
-
+
 (todo: add explainers to all folders and to the most critical files)

docs/images/README.md

@@ -3,6 +3,4 @@ The
 folder holds graphics included in the 
 [architecture document](https://github.com/Constellation-Labs/constellation/docs/architecture.md).
 
-:art: Note that the image repository with branding designs can be found under:
-
-* [/visual-identity](https://github.com/Constellation-Labs/visual-identity)
+:art: Note that the image repository with branding designs can be found under [/visual-identity](https://github.com/Constellation-Labs/visual-identity).

docs/software-dependencies.md

@@ -9,7 +9,7 @@ You'll get a good impression of the used external Scala tools by taking a look a
 
 See
 
-* [/docs/design-choices.md]()https://github.com/Constellation-Labs/constellation/blob/developer/nikolaj/add-docs/docs/design-choices.md)
+* [/docs/design-choices.md](https://github.com/Constellation-Labs/constellation/blob/dev/docs/design-choices.md)
 
 #### :book: References
 

docs/tools-and-frameworks.md

@@ -1,17 +1,15 @@
 ### Tools and frameworks to work with the Constellation protocol
-#### Options
 
+(todo: plaintext explainations of what we use and short note on alternatives):
 
-#### Core team
-
-(todo: plaintext explainations of what we use):
-
-* [wikipedia/Docker_(software)](https://en.wikipedia.org/wiki/Docker_(software))
-* [wikipedia/Vagrant_(software)](https://en.wikipedia.org/wiki/Vagrant_(software))
+* [wikipedia/Docker](https://en.wikipedia.org/wiki/Docker_(software))
+* [wikipedia/Vagrant](https://en.wikipedia.org/wiki/Vagrant_(software))
+* [wikipedia/Google_Cloud_Platform](https://en.wikipedia.org/wiki/Google_Cloud_Platform)
+* [wikipedia/Terraform](https://en.wikipedia.org/wiki/Terraform_(software))
 * [circleci.com/docs/](https://circleci.com/docs/)
+* ...
 
-(tbd)
-
-For a discussion, see:
+#### Discussion
+See
 
-* [/docs/design-choices.md](https://github.com/Constellation-Labs/constellation/docs/design-choices.md)
+* [/docs/design-choices.md](https://github.com/Constellation-Labs/constellation/blob/dev/docs/design-choices.md)

terraform/README.md

@@ -1,53 +1,61 @@
 ## Setup
+Some steps specific to GCP (Google Cloud Platform). For other platforms, this will require changes.
 
-(Some steps specific to GCP, requires changes for other platforms)
-
+```bash
 brew install terraform
 brew install jq
+```
 
 ### Credentials
-
 Locally, make sure this environment variable is set to your google credentials file:
-```export GOOGLE_APPLICATION_CREDENTIALS=${path to your file}```
-[Create and download a new credentials json](https://console.developers.google.com/apis/credentials?organizationId=802489480189&project=esoteric-helix-197319)
-Use your own GCP project (if running a test node outside Constellation org)
-If you don't have a service account, make one with compute engine access,
-otherwise, make a new credentials json file by creating a new service account key and download it to your local machine
 
+```bash
+export GOOGLE_APPLICATION_CREDENTIALS=${path to your file}
+```
 
-### SSH USER
+Use your own GCP project (if running a test node outside Constellation org).
+If you don't have a service account, make one with compute engine access. Otherwise, make a new credentials json file by creating a new service account key and download it to your local machine.
+Here is how to create and download a new credentials json: 
+[Link to Google Cloud API/Credentials](https://console.developers.google.com/apis/credentials?organizationId=802489480189&project=esoteric-helix-197319).
 
-MAKE SURE YOU HAVE EXECUTED
+### SSH USER
+*Make sure you have the following exectued, otherwise you will break the cluster!*
 
+```bash
 ssh-add ~/.ssh/id_rsa
+```
 
-Otherwise you will break the cluster!
+Terraform requires agent forwarding to pick up the keys, it will break the machines due to overwhelming `sshd` slots.
 
-Terraform requires agent forwarding to pick up the keys, it will break the machines due to overwhelming sshd slots
+By default, the terraform command will ask for the ssh user to use. 
+If you'd prefer to avoid answering this every time, you can set the TF_VAR_ssh_user env variable.
 
-By default, the terraform command will ask for the ssh user to use. If you'd prefer to avoid answering this every time, you can set the TF_VAR_ssh_user env variable.
-```export TF_VAR_ssh_user=${USER}``` is probably what you want.
+```bash
+export TF_VAR_ssh_user=${USER}
+```
 
+is probably what you want.
 
-#Initialize
-
+### Initialize
+```bash
 cd to terraform/default
-
-(or copy paste and change settings, make sure to change terraform.backend.prefix in main.tf)
+```
+
+or copy paste and change settings, make sure to change terraform.backend.prefix in `main.tf`.
 
-If you have a pre-existing state already (after copying folder with new name), say 'no' when the prompt asks to copy the state 
-
-and then run
+If you have a pre-existing state already (after copying folder with new name), say 'no' when the prompt asks to copy the state. And then run
 
+```bash
 terraform init # to setup plugins
 
 terraform show # to see existing state (or if it's a new folder should return empty)
+```
 
 ### Usage
-Something something terraform apply / terraform destroy
-
-terraform show # after the cluster is online
+_Something something terraform apply / terraform destroy_
 
-to verify it's working.
+```bash
+terraform show # after the cluster is online, to verify it's working
 
 ./ips_to_hosts_file.sh # to grab ips
+```