rachleona's GSoC

TLDR: GSoC is ending soon and I’ve definitely learned a lot from my time here, if you’re interested in the code I’ve written for my GSoC project feel free to go straight to the end where I’ve linked all the MRs I’ve been involved

Hello GNOME community! Time flies and my time with GSoC working on a new Web-IDE for TinySPARQL is coming to a close. You might have seen my intro post about the project, or the lightning talk my colleague Demigod and I recorded together for GUADEC. In any case, I’m excited to show you guys our final product and talk about the next steps, both in terms of this project and my involvement with open source.

First of all, to reiterate the purpose of this project – we’ve been working the last few months to create a web-IDE to be used with TinySPARQL and LocalSearch for query testing in a more user-friendly environment, our main target audience being fellow developers that for any reasons need to interact with LocalSearch or TinySPARQL databases.

My main work during the last few months involved developing a lightweight TypeScript based UI while my colleague Demigod worked mostly on implementing the backend support necessary.

The first big hurdle of the project for me was figuring out how to include the TS code and necessary npm packages to the TinySPARQL codebase without creating bloat for what is supposed to be a very lightweight and low-level package. We ended up using webpack for bundling and then further compressing them into GResources such that only these GResources need to be included in our releases, to be served when a user starts up the web-ide. This quickly addressed my mentors’ concern about having to include npm packages in our releases and ensured we could work comfortably between the TS code and C backend without any troubles.

In terms of actual design and UX work, this went by relatively smoothly, though it did take quite a few feedback cycles to get it to its current state. Click here for a quick demo of the final product.

Just to go over some of the features that have been fully implemented on the web ide frontend:

• code editor with full support for SPARQL syntax highlighting and common keyboard shortcuts
• error highlighting at corresponding editor positions according to error messages returned by backend
• query “bookmarking” via conversion to links
• Neat table format for presenting query results, equipped with ontology prefix adaptations and directin linking to relevant documentations
• Options to hide/show columns in result tables
• Clear error reporting

And here are some features still in progress/waiting to be merged:

• Options to query in other rdf formats: trig, turtle and JSON-LD
• Examples box for referencing queries that may be useful for certain endpoints
• Quick switching between different SPARQL endpoints from the Web IDE interface itself

In terms of future work, some more work needs to be done on our colour scheme as well as the presentation of query results in the other formats offered other than the default cursor format. There were also some discussions in earlier stages of planning about implementing autocomplete and other editor enhancements that we didn’t have enough time for, so there’s still definitely lots of room for improvement. Nonetheless I’m very satisfied and proud of what I’ve achieved in the past few months and will be looking forward to contributing to future improvements of this tool.

Regarding the overall learning experience, one of the most important thing I learned, in my opinoion, was how to keep my git history clean and work with multiple branches of code at the same time without creating conflicts. I feel like this would completely change the experience of whoever I work with in the future as well as myself when I need to go back to some old code. Other than that, I’ve also had little exposure outside web development and working with the GNOME ecosystem was definitely a nice challenge – I’m definitely a lot more confident about dabbling outside my area of expertise now.

Lastly, here’s a list of useful links to the work I’ve been doing over the summer. Thanks for reading this far

• Main TinySPARQL repo
• Initial MR setting up webpack and the frontend architecture (merged)
• Initial UI setup (merged)
• Error highlighting (merged)
• UI/UX improvement cycle (merged)
• Support for querying results in different formats (open)
• Query examples endpoint (draft)

One of the most recent features I worked on in this GSoC project was to add error highlighting onto the editor if the sparql server returns an error. Parser errors are designed to be returned alongside a byte number where the error originates, so surely that wouldn’t be too difficult right? Well it turned out to involve a bit more engineering than expected.

To start off, the editor we built using CodeMirror cannot be styled directly using CSS, as the styling will not persist over changes in the editor state (e.g. when you start typing or change the position of the cursor). The way this is intended to be done with this package is to use a Decoration object instead. There are different types of decorations including lines and marks, and using the range() method on the Decoration object I would be able to introduce a persisting decoration for a certain byte range in the editor text.

For example, if there is an error on the line starting from byte 16, the following code will produce the decoration I need to highlight the entire line in red:

const errorLine = new Decoration.line({
  class: "redBackground"
}).range(16);

However, this is not enough as the decoration needs to be dispatched to the existing editor view, and the documentations weren’t very clear on how to do this. Luckily, after taking some time to scrutinise this page, I managed to pick out what I need.

There were 2 more key objects I needed from the CodeMirror library: a StateEffect to be dispatched as trigger and a StateField to monitor the effects, producing the error highlights when necessary. The complete code ended up looking something like this:

const errorLine = Decoration.mark({ 
  class: "error-line", // this class gives a red underline
  inclusiveEnd: true
 });

const errorLineEffect = StateEffect.define<{ pos: number }>();
const errorLineField = StateField.define<DecorationSet>({
  create() {
    return Decoration.none;
  },
  update(field, tr) {
    field = field.map(tr.changes);
    let errorLineTriggered = false;

    for (let e of tr.effects) if (e.is(errorLineEffect)) {
      const charAtErrorLine = view.state.doc.toString()[e.value.from - 1];

      // the code below adds the errorLine decoration to the correct positions
      if (charAtErrorLine == "\n") {
        field = field.update({
          add: [errorLine.range(realErrorPos - 1, realErrorPos)]
        });
      } else {
        field = field.update({
          add: [errorLine.range(realErrorPos, realErrorPos + 1)]
        });
      }
      errorLineTriggered = true;
    }

    // remove the decoration once editor text changed as error may no longer exist/may now be in a different position
    if (tr.docChanged && !errorLineTriggered)
    {
      field = field.update({
        filter: (f, t, value) => !value.eq(errorLine)
      })
    }
    return field;
  },
  provide: f => EditorView.decorations.from(f)
})

// function to trigger errorLine application
export function setErrorLine (pos: number) {
  let effects: StateEffect<unknown>[] = [ errorLineEffect.of({ pos }) ];

  if (!view.state.field(errorLineField, false)) 
    effects.push(StateEffect.appendConfig.of([errorLineField]))
    
  view.dispatch({ effects });
}

And this works as intended, producing the following in the the interface:

But there was one final problem, this does not work for strings of all encoding, e.g. a query like this: select ('👩🏿‍🦱👩🏽‍🦳' AS ?str) { asdf } will trigger an error response at the wrong position. Luckily, this was a relatively simple fix using TextEncoder/Decoder in JS, as demonstrated with our example query below:

Hi GNOME community! It’s been a while I caught COVID a while back so it took some time to catch back up on work but we’ve been making progress on the GSoC project. We now have a fully functional IDE with a smooth interface and syntax highlighting – take a look!

It is also equipped with basic error reporting functionalities.

With the basic structure and functionalities complete, our next steps will be to extend the error response to also reflect the line from which the error is originating from, as well as implementing a bookmarking feature to allow users to “save” their queries using links.

There have also been suggestions to create an RDF browsing interface on the project. This will be a challenge to implement as it would require quite a lot of flexibility in the display, but given that we still have quite a bit of time until the end of GSoC, it is a challenge I hope I would be ale to take on.

Greetings, GNOME community! My name is Rachel and I’m currently studying computer science at the University of Cambridge. I’ve been looking for an opportunity to learn to contribute to the open source community and am super excited to be working with GNOME as a gsoc contributor this summer

I’ll be working with my mentors Carlos Garnacho and Sam Thursfield, as well as my fellow contributor, Divyansh Jain (aka Demigod) in creating a web IDE for testing and running queries in TinySPARQL, as an effort to improve developer experience when working with TinySPARQL. I will be using this blog to update the community about our project and my current tasks, so please look forward to our work. Bye for now!