Skip to content

ETL to QE, Update 17, Readjusting Goal Posts

2023-10-29

Let's review what the plan for ETL to QE was,

  1. Discord Analytics Reports and Dashboard
  2. Graph Based Annotation on Top of Discord Data
  3. Allow for Generalized Questioning and add Additional Data Sources
  4. Proof of Meme Micro Bounty Platform

Each of these steps was supposed to be completed in one week yet here we are 4 weeks later still stuck on step one.

There are a series of problems that have lead to this predicament including, 1. The goal of this project never being expressed 2. No measurable milestones to judge where I am relative to my goals 3. No third party to judge my work and ideas 4. No intended first user 5. No explicit problem statements 6. Not enough research done into the problems

All the problems I just listed not only effect my this project by my blog and writings as well. The other day I realized I have not written a full proper essay since collage. I have plenty of half finished essays in Backlog - Writing even the stuff I have published in Blog Posts is not even finished.

There were and are a lot of assumptions regarding the trajectory my blog and ETL to QE project were supposed to go. It is time I think through them so I have some goal posts I can actually aim towards.

Let's keep the focus on ETL to QE.

Recreate Keybase Analytics Dashboard

For step 1. Discord Analytics Reports and Dashboard I was supposed to recreate the dashboard I did for the Keybase Binding with the source code linked here but for Discord Data. But I had never gotten to really developing the dashboard because I never decided on a list of queries I wanted to implement. I do set of queries such as, Questions for Discord Data, and even specific sets of questions such as, Queries Comparing Discord Guilds, but I never got around to creating an user interface for them.

One issue with all those queries and questions I wrote about is that they have questionable utility. I can see how many messages someone posted. That metric can't really be acted on. I can see the longest messages from a discord guild or user. But that data is also useless unless I go read the message and add some context.

I have always lived with the assumption that I would get around to building the interface to layer context on top of the discord data but never explicitly wrote that as my goal. Like what does it mean to layer context on top of data.

Conduct Example Investigation

I currently have a note in all caps REMEMBER TO DO EXAMPLE INVESTIGATION sitting in my Discord Binding document. I have been working on this project for literal weeks and have yet to do an actual example investigation. I never wanted to get specific even though constraints, specifics, is where all the value is at.

This Discord Binding project has always had a couple assumed questions like,

Conduct Basic Research into DAOs

I also have my Questions for DAO's document which has a series of questions that I believe every DAO should be able to answer. I still have yet to answer those questions for any DAO. I should be using my analysis of the DAO data to find people that can answer those questions.

Topic Modelling was supposed to be easy

I initially assumed that I would just run Topic Modelling on each discord channel would be easy and consist of the following steps,

  1. Feed in 1000's of individual messages from channel
  2. Get list of key word topics out
  3. Store results in database
  4. Repeat for next channel or user

After this was completed the resulting data was supposed to reveal some kind of insights. I did not go in with any expectation, not even a hypothesis. I just thought I might see some interesting topics that would lead to specific channels and individuals worth engaging in.

Average Message Length would be an important indicator

I still have yet to find the user with the highest average message length. I should also probably require a minimum of say 20 or 100 messages.

Get money from DAOs

My goal subliminally has always been to build reputation within DAO's completing different projects and tasks until I became some sort of journeyman or master.

I want to find my Master

I really want to have some sort of mentor in my life. I want someone to sit down with me and interrogate the shit out of my projects. I want someone who has the nuanced background so that they can connect me with clients once I have the right expertise. I want to be able to hang out over tea or beers and shoot the shit.

I should be able to list 10 important individuals from DAO's off the top of my head

I should articulate my vision for where I think DAO's should be going

I should have a model for knowing what is worth knowing

I need Question Engine up and running so I can actually do proper research on these DAO's.

Question Engine turns something like wikipedia into something as engaging as Discord or Reddit.

We can do an example with Maker DAO Community

Mimetic Supply Chain

I am here to produce intelligence

What I am trying to do is develop an intelligence tool. I guess almost all software on some level counts as a intelligence tool in one way or another.