Encoding

CCL's Principal Analyst Alex Caithness asks the question: After SQLite, what comes next? A must-read primer on LevelDB - tomorrow's ubiquitous format?

The personal website of Taras Gritsenko.

Modern SPA frameworks add drawbacks to workflows. Inertia.JS allows building modern SPAs, easily integrating with backend frameworks.

It covers how to use the standard encoding/binary package to encode binary according to a custom format, and how it works

It covers how to make a time-series database engine from scratch

The design of the Internet Computer utility token treasury.

The Google.Protobuf NuGet package is the recommended protobuf serializer for .NET. The documented way of using it is to code gen both C# models and serializers from .proto files using the protoc tool. However, sometimes it’s more convenient to do serialization/deserialization on an ad-hoc basis without code generation. The Google.Protobuf NuGet package provides APIs to do this, but they are poorly documented. This post gives a code example for a simple “no-code-gen” serializer.

Protoscope is a simple, human-editable language for representing and emitting the Protobuf wire format. - protocolbuffers/protoscope

Quoting from the README: Postcard is a #![no_std] focused serializer and deserializer for Serde. Postcard aims to be convenient for developers in constrained environments, while allowing for flexibility to customize behavior as needed. I first published postcard back in 2019, as a way to get "all the good stuff from Serde" in a format that would work for embedded systems. Since then, people all over Rust are using Postcard as a general purpose, compact, and flexible Serde format, not just embedded folks! Now, 3 years later or so, it's about time to take it to v1.0. Thanks to a bunch of accumulated experience over the years, and a generous sponsor, I'll be releasing the 1.0 version of Postcard in June of 2022. Read on for more details on what's planned!

Leaky abstraction in the Protocol Buffers wire format

Status: Stable for the trace, metric and log signals. Development for the profiles signal. The OpenTelemetry Protocol (OTLP) specification describes the encoding, transport, and delivery mechanism of telemetry data between telemetry sources, intermediate nodes such as collectors and telemetry backends. OTLP is a general-purpose telemetry data delivery protocol designed in the scope of the OpenTelemetry project. Protocol Details OTLP defines the encoding of telemetry data and the protocol used to exchange data between the client and the server.

I was recently rewarded a total of $107,500 by Google for responsibly disclosing security issues in the Google Home smart speaker that allowed an attacker within wireless proximity to install a "backdoor" account on the device, enabling them to send commands to it remotely over the Internet, access its microphone feed, and make arbitrary HTTP requests within the victim's LAN (which could potentially expose the Wi-Fi password or provide the attacker direct access to the victim's other devices). These issues have since been fixed.

How I designed and implemented the Object Hash Set, a high-performance hash set for Javascript objects

Package binary implements simple translation between numbers and byte sequences and encoding and decoding of varints.

An effort to come up with a compressed binary vector geometry specification - TWKB/Specification

Package binary implements simple translation between numbers and byte sequences and encoding and decoding of varints.

Fast Encoding Using Pre-serialization Protocol Buffers is a popular serialization mechanism with implementations available for most major languages that allows to easily model complex data structures and has good performance characteristics. However, for performance critical applications one always wishes for faster encoding and decoding, especially when the application performs minimal processing and serialization dominates the performance profile. This post shows how it is possible to increase encoding and decoding speed several times for certain special cases.

Parquet encoding definitions This file contains the specification of all supported encodings. Unless otherwise stated in page or encoding documentation, any encoding can be used with any page type. Supported Encodings For details on current implementation status, see the Implementation Status page. Encoding type Encoding enum Supported Types Plain PLAIN = 0 All Physical Types Dictionary Encoding PLAIN_DICTIONARY = 2 (Deprecated) RLE_DICTIONARY = 8 All Physical Types Run Length Encoding / Bit-Packing Hybrid RLE = 3 BOOLEAN, Dictionary Indices Delta Encoding DELTA_BINARY_PACKED = 5 INT32, INT64 Delta-length byte array DELTA_LENGTH_BYTE_ARRAY = 6 BYTE_ARRAY Delta Strings DELTA_BYTE_ARRAY = 7 BYTE_ARRAY, FIXED_LEN_BYTE_ARRAY Byte Stream Split BYTE_STREAM_SPLIT = 9 INT32, INT64, FLOAT, DOUBLE, FIXED_LEN_BYTE_ARRAY Deprecated Encodings Encoding type Encoding enum Bit-packed (Deprecated) BIT_PACKED = 4

This post introduces Arrow Flight, a framework for building high performance data services. We have been building Flight over the last 18 months and are looking for developers and users to get involved.

Breaking down and investigating how Dark Souls 3 communicates with its online services.

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values. - google/leveldb

Some of you may know I've been spending whatever time I can scrounge together grinding away at a new serialization library for .NET. Serializers can be complicated beasts. They have to be reliable, flexible, and fast beyond reproach. I won't convince you that serialization libraries have to be quick — in this post, that's a given. These are some tips from my experience in optimizing Hagar's performance. Most of this advice is applicable to other types of libraries or applications.

In a recent article, I presented rules for evolving JSON Schemas in a backward, forward, and fully compatible manner. The backward compatibility rules for Protocol Buffers (Protobuf) are much simpl…

Learn how to transfer data efficiently across networks by using Protocol Buffers in Python

This document provides a gentle introduction to the data structures and formats that define the certificates used in HTTPS. It should be accessible to anyone with a little bit of computer science experience and a bit of familiarity with certificates. An HTTPS certificate is a type of file, like any other file. Its contents follow a format defined by RFC 5280. The definitions are expressed in ASN.1, which is a language used to define file formats or (equivalently) data structures. For instance, in C you might write:

We’re excited to announce support for Swift in Wire. Wire already supports compiling your protocol buffer files into Java and Kotlin, and today Swift joins that family.

Write an awesome description for your new site here. You can edit this line in _config.yml. It will appear in your document head meta (for Google search results) and in your feed.xml site description.

Package binary implements simple translation between numbers and byte sequences and encoding and decoding of varints.

This is #5 in a very long series of posts on Stack Overflow’s architecture.Previous post (#4): Stack Overflow: How We Do Monitoring - 2018 EditionSo…cachin...