Development – dave-gill.co.uk

GMPS

So my new project posts are now going to be called GMPS posts because that is what I wanted to call the thing.

I have finished most of the boring crud stuff around registering users, inviting users and creating groups etc. It was at this stage I stopped for a while thinking about what I should be doing now. The obvious next step was to get my Syanpse server up and running and attempt to link my Laravel app through some internal calls to it that would allow me to register a user.

My first attempt at this was to use the GuzzleHttp libraries that come with Laravel. I built a very quick and dirty piece of code to see if I could get that working following the guide from the matrix.org docs on the client API. That at least got me to the stage where I was getting a response from the Synapse server, great start. I was only getting the Unauthorized 401 response and had to dig through the JSON returned to find more information. It was sending back a session ID so I tried to call it twice, once to get a session ID and once with that session ID. Still no joy, still the same response. I tried changing some of the parameters I was sending through Guzzle but nothing seemed to be working. So I stopped trying for a while and thought I would look around and see if someone had done anything similar with PHP in the past.

On my little Net Trawling I came across a Github repo that had built a Laravel package (Perfect!) that was doing what I wanted to do. Now it had not been maintained for 3 years but I thought it would at least be a starting point so I forked the repo and had a good look at the code. I had a choice of whether to install the package as-is or take the code and adapt it into my code base. I did think if I left it as a package then it could be maintained separately from my own code. But, I thought I would learn more if I took the code and wove it into my own project. So that is what I am doing.

The original code had a ServiceProvider and a class called Matrix. The code was aimed at an an older version of Laravel I think and plus I was not keen on the manner it was drawing in config details so I created my own ServiceProvider and started to lift and shift some of the code across for the registration process. And that is where I got my first bit of joy actually getting the Synapse server to register a user and get a userid and token for them to use. Brilliant, now I have to implement all the other things.

New Project – 4

In my last Post I said I would attempt to create the project using RUST. Well after playing with RUST and Rocket for a few days I came to realise the naivety of that approach. RUST and Rocket are both great but do not have the tooling and plugins available to make this a quick simple job, so I have abandoned that approach to go back to PHP and Laravel.

On that note I thought I had given the project enough thought pre-coding for me to get straight in and crack on with minimum fuss. Nope. Again my naivety shows up again. I was happily writing the code for inviting participants to groups and realised I felt a bit uncomfortable about the thing, i.e. I had the faint whiff of something not 100% correct. Now being fairly pragmatic I recognised this as a good time to stop and look at what I was building. That meant no more code until I could identify the smell and find ways to get rid of it.

Hopefully I have now done that but I am going to give it another day of thinking and write down my thoughts before I start coding again. Hopefully I will have truly understood what the smell was and redesigned my way out of it.

After all, this is my project, my code and I can rewrite it as many times as I like.

New Project – 3

Now that I have Synapse running on my laptop I thought it would be good to start building out the client that I am going to try and build for it. I have been writing PHP code for a few years now and my initial thought and desire to get coding led me to start building the front end in PHP making use of Laravel.

In fact I got as far as putting together the register pages and the login pages. Not too difficult in Laravel, it provides tools to do most of the work for you. And then I went away for a couple of days without my laptop and left the idea to gestate a while. I then thought it would be better to write it in RUST. I have a plan to learn a new language each year and this year was RUST, so it would make sense to build it in the language I am trying to learn. Yes it will take a hell of a lot longer and will be more frustrating I am sure. But there are 2 major benefits, 1 The best way for me to really learn a language is to build something with it. 2 The finished result should be so much faster than the PHP version.

So I am going to start the code again in Rust.

While away I also thought of a name for the project which is just an Acronym of what I am trying to achieve by doing this. Generic Matrix.org Permission System or GMPS. I want to build something on top of the matrix.org network that will allow an organisation to pick up the project and implement a matrix.org network but with finer grained controlled of who can access what.

New Project – 2

So the first step in my new little project is to get a Matrix Homeserver up and running. Without one I can not start to build a client for it. So the only real choice with this was to use Synapse a Python based server. To do this I wanted to build it locally so that I can play and not interfere with anything else on the internet.

The ansible-docker playbook was too tempting for me to avoid so I thought I would start with that as it covered everything you would want to install to get things fully functional. Because I was trying to run this on my local machine and not on a server I had a few problems with this, the first of which was the version of Ansible I had on my laptop. I got quite far through the process and then hit a problem with certbot not being able to provide a certificate because it was unable to register. Now this had taken me a while to get this far and I suddenly realised I did not need the all singing all dancing implementation. 1 Because the project was supposed to be fun and the fun had stopped and 2 I was trying to build a client for the Homeserver and did not need everything else to do that, so I stopped on that track and just decide to install Synapse on its own locally.

This was as simple as following the instruction on the github repo for synapse. Once you have installed the pre-requisites you can get a prebuilt package. So I had Synapse installed (but not running) in a matter of minutes. Ran the config generator and started the server with zero problems ( Ok that’s a Lie. I had a mistype in one of the filenames). I then check on the local server to see if it was running.

So my next step was to create a user, again using the instructions from the github repo and I had a user. Now I have no idea what to do with it. The next stepis to learn the API and see if I can hack together a client of some sort.

New Project

So I have decided to start a new open source project to build a communication platform for any type of organisation. Think Slack but simpler and perhaps a bit more focused on a niche.

At first I thought of taking the same kind of approach as Slack and just basing everything off IRC and putting a nice facade over the top, but then that is simply doing the exact same thing, which is a bit boring really. I did go down the route of setting up an IRC server and writing a few scripts to automate the joining of channels and accepting of messages etc., but why re-invent the wheel when Slack have done such a good job.

I then thought I could do something similar to my day job but then you get into all sorts of problems with IP rights and source code.

So instead I am going to look at doing it using the work done by Matrix. Which will provide me with a couple of things really useful to a new project:-
1. It is a fairly new piece of technology so I will be learning something new.
2. All of the communication infrastructure code has been written I can do a Slack and just piggy back off it.
3. I have no idea what I am doing, which is always a great place to start.

I am going to keep this blog updated with the progress, if I make any at all.

Clojure – Closure

I tried very hard to go through the books I have for Clojure and event go to the point of thinking some of the concurrency and parallel programming stuff was really very cool. But I have been a way for a bit and read nothing on Clojure and feel like I have got to that point where I will stop for a while.

My Masters starts up again soon and Ireally do not want to screw up my dissertation so I might go back to it when I have completed that. But then perhaps I wont. It is so different to anything I have ever tried to do.

Clojure

So I am a firm believer in trying to learn a new computer language each year and so this year I am trying Clojure. And trying is probably the best I can do.

The last time I used a Lisp was back at University and rarely got above trivial. So I have come to it almost fresh. I have to say the concepts are fine and I like the idea of functional programming and the immutability but as soon as it gets away from the elementary level I struggle to get my head around it all.

I have bought books (4 in fact) and have been trying theclojure koansand also the 4clojure web site but still struggle with anything from the more difficult sections.

So at the moment I am the ignorant apprentice waiting for the “ah ah” moment of Clojure enlightenment. I am yet to get to the despondent stage where I give up but feel it is not too far away.

Node.Js worries me

There are a couple of things that concern me about it and the most pressing is security. It is just so easy to include hundreds of packages the source of which is almost impossible to track.

If those modules are maintained by people you don’t know anything about how do you know they can be trusted. How do you know if someone has not changed the code somewhere in those packages to do something malicious.

But that concern was precipitated by my very first concern. The sheer number of packages you can install without even trying to do so. Next time you play with node just check how many packages youhave just by including express and a couple of components.

And then finally perhaps the reason that appears to be the most churlish is that just because anyone can now write backend code with JS does not mean that you should. Sometimes Node just does not do the trick. If you want to create a system that processes and transforms lots of data, Node is not the right tool.

All the technologies

Do people really think that throwing every new piece of cool technology at a project will make it a success? I love new technology and used to spend time playing with new Frameworks etc so I could find a use pattern for them. But very few of them made them into projects because their was no value to be added from doing so.

I have seen so many projects where the business case for building them is based on keeping a technology stack up to date. Great if that is going to add value or provide a competitive advantage. But spending time and money to move your website to the latest and greatest JS framework is not a valid reason.

Best practice in web development???

This is not a blog post to show what is best practice in web development but a rant against those people who use the term ‘best practice’ to defend their use of a technology.

I shall explain with a couple of examples. Yesterday at work a developer said the business could not have a web page with 5 even columns on the page because they use bootstrap which uses a 12 column grid. I tried to point out this was an issue and was told using Bootstrap was ‘best practice’.

We are building a new website and it HAS to be hosted on AWS because it is ‘best practice’.

We are building a new website in PHP and we have to use a framework because that is ‘best practice’.

I have nothing against Bootstrap, AWS or PHP Frameworks but this highlights that people are building a solution before they have even looked at the requirements. Each project is unique and has unique requirements so to come to the conversation with a fixed mindset of what solutions will work is just plain wrong.

It could be that these solutions are the best thing for the project but you can’t decide that because it is ‘best practice’. You have to decide that because it is best for the project.