commit 518e430d6124ae69c75c787eb9ff02eb7cf5c1a4 Author: Sanchayan Maity Date: Fri May 10 15:33:13 2024 +0530 Start note taking diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..087e483 --- /dev/null +++ b/.gitignore @@ -0,0 +1,5 @@ +private.md +private-*.md +*.odt +*.docx +.obsidian diff --git a/android.md b/android.md new file mode 100644 index 0000000..52a8530 --- /dev/null +++ b/android.md @@ -0,0 +1,260 @@ +--- +title: Android +--- + +# Binder + +## Reference material + +- [Deep Dive into Android IPC/Binder Framework](https://www.youtube.com/watch?v=Jgampt1DOak) + +- [AnDevCon IV: Android Binder IPC Framework](https://www.youtube.com/watch?v=hiq3mGfLOtE) + +- [Digging into Android System Services](https://www.youtube.com/watch?v=M6extgmQQNw) + +- [Binder transactions in the bowels of the Linux kernel](https://www.synacktiv.com/en/publications/binder-transactions-in-the-bowels-of-the-linux-kernel.html) + +## Notes from deep dive + +- An IPC framework for developing object oriented OS services. + +- Built in reference counting mechanism. + +- Identifying senders to receivers via UID/PID. + +- Unique object mapping across process boundaries. + +- Ability to send file descriptors across process boundaries. + +- Android supports a simple form of IPC via `intents` and `content providers`. Uses binder behind the scenes. Asynchronous. + +- Messenger IPC + + - `Messenger` represents a reference to `Handler` that can be send to a remote process via an `Intent`. + - A reference to the `Messenger` can be sent via an `Intent` using the previously mentioned IPC mechanism. + - `Messages` send by the remote process are delivered to the local handler. + - `Messages` are like `Intents` in that they designate the operation and data. Still asynchronous. + +- `Parcel` is a container for a message (data and object references) that can be sent through an `IBinder`. A unit of transactional data - one for the outbound request and another for the inbound reply. + +- `Marshalling/Unmarshalling` + + - Marshalling is a procedure for converting higher level app data structures i.e request response parameters into parcels for the purposes of embedding them into Binder transactions. + + - Unmarshalling is a procedure for reconstructing higher level app data structures i.e request response parameters from parcels received through Binder transactions. + +- Proxy is an implementation of the AIDL interface that un/marshals data and maps method calls to transactions submitted via a wrapped IBinder reference to the Binder object. + +- Stub is a partial implementation of the AIDL interface that maps transactions to Binder service method calls while un/marshalling data. + +- `ContextManager` or `servicemanager`, a special binder object with a known handle (registered as handle 0) that's used as a registry/lookup service for other Binder objects. + +- Most low level operations and data structures i.e `Parcel` are abstracted by `libbinder` at the native level which is what the clients and services use. + +- Clients and services don't want to know anything about the `Binder` protocol and `libbinder`, so they makes use of `proxies` and `stubs`. + +- `Proxy's` job is to take the high level Java or C++ request, and convert it into one of `Parcels` and then submit an `ioctl` transaction to the binder driver. `Stub's` is to listen to the binder driver callback, and then upon receiving a callback unmarshal the `Parcel` into something the service can understand and call the appropriate callback in the service. Java based proxies and stubs can be automatically generated by `aidl` tool for services described with AIDL. Service needs to implement the binder interface. + +- `Manager` sits between the `Client` and `Proxy`. `ContextManager` sits between the binder driver and the service. + +- Anything you pass as an `IBinder` to the other side gets passed as a reference. + +## Binder object reference mapping across process boundaries + +- A binder object reference is one of the following + + - An actual virtual memory address to a binder object in the same process + + - An abstract 32-bit handle to a binder object in another process + +- On every transaction, the binder driver automatically maps local addresses to remote binder handles and remote binder handles to local addresses + +- This mapping is done on: + + - Targets of binder transactions + + - `IBinder` object references shared across process boundaries as a parameter or a return value (embedded in transaction data) + +- For this to work + + - The driver maintains mappings of local addresses and remote handles between processes as binary tree per process so that it can perform this translation + + - References embedded in transaction data is discovered based on effects that the client provies when it submits its transaction and then rewritten in place + +# Thumbnail extraction + +[Framework Hardening](https://source.android.com/devices/media/framework-hardening) + +- Abstract class `Thumbnailer` in `MediaProvider`. Note that `MediaProvider` includes `ThumbnailUtils`. A separate directory for thumbnails is used `.thumbnails`. Creates an audio thumbnail for music directory, video thumbnail for movies directory and image thumbnail for pictures directory. Bitmap format is used for the thumbnails. Note that these utilities functions have a note mentioning that these should be used only if direct access to it's available. If it's media hosted outside app, use `ContentResolver#loadThumbnail` which is in `ContentResolver.java`. This seems to be the way `MediaStore` uses it, though the function where it's used seems to be deprecated. + +- Ensures that thumbnails collection on the given storage volume can be used with the given database. If the database UUID doesn't match the UUID found on disk, then all thumbnails are considered stale and deleted. Seems to get called in `onIdleMaintainance`. + +- Primarily three functions are seen which are called in `MediaProvider` from `ThumbnailUtils`. + + 1. `createAudioThumbnail` + + 2. `createVideoThumbnail` + + 3. `createImageThumbnail` + +- These create an instance of `MediaMetadataRetriever` and then call methods on this instance. For example, `createVideoThumbnail` calls `extractMetadata`, which then calls in to the `nativeExtractMetadata`. The native implementation itself calls the implementation in `MediaPlayerService` through a Binder call. See the constructor in `mediametadataretriever.cpp`. + +- `MediaProvider` seems to primarily use the `invalidate` and `ensure` thumbnail functions. The `Thumbnailer` instances created for audio, video and image primarily call `ensureThumbnail` from the `Thumbnail` abstract class. This function has a `ParcelFileDescriptor`. + +- `FileDescriptor` objects, representing raw Linux file descriptor identifiers can be written and `ParcelFileDescriptor` objects returned to operate on the original file descriptor. + +- The flow in `MediaProvider` is `openFile -> openFileCommon -> ensureThumbnail -> mAudio/Video/ImageThumbnailer.ensureThumbnail`. + +- The actual extraction of the thumbnail happens in `getBitmapFromVideoFrame` in the native code. + +### Summary + +`MediaProvider` uses the `MediaMetadataRetriever` which eventually relies on native code via JNI to extract the thumbnail. This seems to be allowed only because the provider here is kind of owner so to speak and has direct access to the media library?. Elsewhere like `MediaStore` or other applications the appropriate interface would `ContentResolver`. + +## Examples + +- https://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/tests/examples/snapshot/snapshot.c + +- https://stackoverflow.com/questions/15789652/how-to-create-video-thumbnails-with-python-and-gstreamer + +# ModernMediaScanner + +- Implements the `MediaScanner` interface below. + + ``` java + public interface MediaScanner { + public static final int REASON_UNKNOWN = MEDIA_PROVIDER_SCAN_OCCURRED__REASON__UNKNOWN; + public static final int REASON_MOUNTED = MEDIA_PROVIDER_SCAN_OCCURRED__REASON__MOUNTED; + public static final int REASON_DEMAND = MEDIA_PROVIDER_SCAN_OCCURRED__REASON__DEMAND; + public static final int REASON_IDLE = MEDIA_PROVIDER_SCAN_OCCURRED__REASON__IDLE; + + public Context getContext(); + public void scanDirectory(File file int reason); + public Uri scanFile(File file, int reason); + public Uri scanFile(File file, int reason, @Nullable String ownerPackage); + public void onDetachVolume(String volumeName); + } + ``` + +- It starts by populating metadata based on file attributes and then overwrites with any valid metadata found using `MediaDataRetriever`, `ExifInterface`, `XmpInterface`. + +- `ModernMediaScanner` is called by `MediaProvider` and gets passed a `Context`. `getContext` just returns this `Context` when called. + + ``` java + /** + * Map from volume name to signals that can be used to cancel any active + * scan operations on those volumes. + */ + @GuardedBy("mSignals") + private final ArrayMap mSignals = new ArrayMap<>(); + + /** + * Holder that contains a reference count of the number of threads + * interested in a specific directory, along with a lock to ensure that + * parallel scans don't overlap and confuse each other. + */ + private static class DirectoryLock { + public int count; + public final Lock lock = new ReentrantLock(); + } + + /** + * Map from directory to locks designed to ensure that parallel scans don't + * overlap and confuse each other. + */ + @GuardedBy("mDirectoryLocks") + private final Map mDirectoryLocks = new ArrayMap<>(); + ``` + +- `scanDirectory` and the two `scanFile` functions call the private class `Scan`. See `ContentProvider`, `ContentResolver` and `ContentInterface`. + +- `ScanItemAudio` and `ScanItemVideo` use `MediaMetadataRetriever`. + +## Working of \`Scan\` which is a public class + +- Get the content provider client using the content resolver + +- The primary function is `run` + + - This calls `walkFileTree`. Acquire directory lock at the start of operation and release at the end. This uses the existing `walkFileTree` from `Files`. The rough way this works is, it's recursive. The file visitor function `visitFile` gets called for every file and depending upon the `FileVisitResult` the operation continues or stop. `visitFile` is what calls `scanItem` which in turn calls the media specific scan functions. Of the media specific scan functions, only the audio and video ones use `MediaMetadataRetriever`, the other uses XMP, ISO, EXIF or standard file utilities. Here something called as `ContentProviderOperation` exists. It represents a single operation to be performed as part of a batch of operations. See the `ContentProviderOperation` section below. + + - Then `reconcileAndClean`. This talks to the database and some operations on the database entries. It cleans up unknown entries and update the generation associated with a file column. + + - `resolvePlaylists`. Validates the playlist members? + +## Media modules documentation + +[Media Modules](https://source.android.com/devices/media/media-modules) + +## MediaStore + +- It's the contract between the `MediaProvider` and applications. Contains definitions for the supported URIs and columns. + +- A generation value is associated with a given volume. Generation numbers are useful for apps that are attempting to identify exactly which media items have been added or changed since a previous point in time. Generation numbers are monotonically increasing over time, and can be arithmetically compared. + +## MediaProvider + +- The MediaProvider module optimizes indexed metadata (audio, video, and images from SD cards and USB devices) and makes that data available to apps through the `MediaStore` API. + +- In this case, `MediaProvider` extends `ContentProvider`. + +- Some of the modules which reference `MediaProvider` are `MediaService`, `ExternalStorageServiceImpl`, `PermissionActivity`, `FuseDaemon` and `IdleService`. + +- For permissions, see `checkUriPermission`. + +### Notes on threading + +- Some of the functions seemed to be called in JNI context and have the note `Called from JNI in jni/MediaProviderWrapper.cpp`. These functions seem to have the current thread in their `JNIenv`. + + ``` java + void MediaProviderWrapper::ScanFile(const string& path) { + JNIEnv* env = MaybeAttachCurrentThread(); + scanFileInternal(env, media_provider_object_, mid_scan_file_, path); + } + ``` + +### Notes on security + +- [Media Provider](https://source.android.com/devices/media/media-provider) + +- [Scoped Storage](https://developer.android.com/training/data-storage#scoped-storage) + +- To maintain privacy, the `MediaProvider` module enforces the `scoped storage security module`. + +## ContentProvider + +- Content providers are on of the primary building blocks of Android applications, providing content to applications. They encapsulate data and provide it to applications through the single `ContentResolver` interface. A content provider is only required if you need to share data between multiple applications. For example, the contacts data is used by multiple applications and must be stored in a content provider. If you don't need to share data amongst multiple applications you can use a database directly via `android.database.sqlite.SQLiteDatabase`. + +- `MediaProvider` extends `ContentProvider`. + +- See more on Content Providers in the below link. + + - [Content Provider Basics](https://developer.android.com/guide/topics/providers/content-provider-basics) + + - [Content Providers](https://developer.android.com/guide/topics/providers/content-providers) + +- When a request is made via a `ContentResolver` the system inspects the authority of the given URI and passes the request to the content provider registered with the authority. The content provider can interpret the rest of the URI however it wants. + +- Requests to `ContentResolver` are automatically forwarded to the appropriate `ContentProvider` instance, so subclasses don't have to worry about the details of cross-process calls. + +- See permissions based on `Intent`. + +### Data access via intents + +Intents can provide indirect access to a content provider. You allow the user to access data in a provider even if your app doesn't have access permissions, either by getting a result intent back from an app that has permissions, or by activating an app that has permissions and letting the user do work in it. + +You can access data in a content provider, even if you don't have the proper access permissions, by sending an intent to an app that does have the permissions and receiving back a result intent containing "URI" permissions. These are permissions for a specific content URI that last until the activity that receives them is finished. The app that has permanent permissions grants temporary permissions by setting a flag in the result intent: + +- Read permission: `FLAG_GRANT_READ_URI_PERMISSION` + +- Write permission: `FLAG_GRANT_WRITE_URI_PERMISSION` + +## ContentProviderOperation + +Used for batch access, see the preceding `content-provider-basics` article link. + +[Better Performance with ContentProviderOperation](https://www.grokkingandroid.com/better-performance-with-contentprovideroperation/) + +## MediaMetadataRetriever + +- For retrieving meta data when it comes to audio and video files, an instance of this is used by `ModernMediaScanner`. It seems to call into the native media extractor via JNI. diff --git a/anime.md b/anime.md new file mode 100644 index 0000000..31524ab --- /dev/null +++ b/anime.md @@ -0,0 +1,22 @@ +## Watching + +- [ ] Saiki +- [ ] Magus Bride +- [ ] Call of the night +- [ ] Heavenly Delusion +- [ ] Dangers in my heart + +## To watch + +- [ ] Toradora +- [ ] Romantic Killer +- [ ] Skip and Loafer +- [ ] Hyouka +- [ ] Mushishi +- [ ] Insomniacs After School +- [ ] Ikoku Nikki +- [ ] Moribito Guardian of Spirit +- [ ] Yona of Dawn +- [ ] Legend of Arslan + + diff --git a/audio.md b/audio.md new file mode 100644 index 0000000..aef6b54 --- /dev/null +++ b/audio.md @@ -0,0 +1,73 @@ +--- +title: Audio +--- + +# Bluetooth + +## Bluez + +- Using \`gdbus\` to introspect or retrieve properties + +``` bash +gdbus introspect --system --dest org.bluez --object-path /org/bluez/hci0/dev_6C_C4_D5_6C_CF_19 +gdbus call --system --dest org.bluez --object-path /org/bluez/hci0/dev_6C_C4_D5_6C_CF_19 --method org.freedesktop.DBus.Properties.GetAll org.freedesktop.DBus.Properties +``` + +## Information on codecs + +- https://www.soundguys.com/understanding-bluetooth-codecs-15352/ + +- https://habr.com/en/post/456182/ + +- https://www.nytimes.com/wirecutter/blog/what-you-need-to-know-about-bluetooth-audio/ + +- https://www.bluetooth.com/learn-about-bluetooth/recent-enhancements/le-audio/le-audio-specifications/ + +- https://www.bluetooth.com/blog/best-recorded-bluetooth-presentations-from-2020/ + +## Transport payload sizes observed for SBC + +- SBC-XQ @ 552 Kbps: 622 bytes +- SBC-XQ @ 512 Kbps: 574 bytes +- SBC-XQ @ 453 Kbps: 678 bytes + +# Random + +## Clean up audio with Audacity + +Consider an mkv file recorded with OBS, do the following + +- Extract the audio from video file + + ``` bash + ffmpeg -i infile.mp4 -acodec copy outfile.aac + ``` + + The option -acodec copy implies to not re-sample the audio. + +- Use Audacity's noise reduction filters. Export as m4a. + +- Merge audio and video again + + ``` bash + ffmpeg -i infile.mp4 -i denoised-audio.m4a -vcodec copy -acodec copy -map 0:0 -map 1:0 denoised-video.mp4 + ``` + +- Increase the volume using ffmpeg + + ``` bash + ffmpeg -i infile.mkv -vcodec copy -filter:a "volume=5.000000" infile-outputlouder.mkv + ``` + + Note that this command seems to change the format in mkv file from aac to ogg. + + To keep the codec same namely AAC, use the following + + ``` bash + ffmpeg -i infile.mp4 -vcodec copy -filter:a "volume=3.000000" -c:a aac -b:a 320k infile-outputlouder.mkv + ``` + + See the link + + + diff --git a/bookmarks.md b/bookmarks.md new file mode 100644 index 0000000..f98ba17 --- /dev/null +++ b/bookmarks.md @@ -0,0 +1,40 @@ +--- +title: Bookmarks +--- + +# Interesting projects + +- https://c9x.me/ +- https://www.gnu.org/software/mes/ + +# Mathematics + +- https://abakcus.medium.com/the-best-of-springer-undergraduate-books-for-beginners-c0c55f9fdb11 + +- https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/index.htm + +- https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab + +- https://twitter.com/thingskatedid/status/1335013038429065216 + +# Computer architecture + +## Course by Onur Mutlu at ETH Zurich + +- https://safari.ethz.ch/architecture/fall2020/doku.php + +- https://www.youtube.com/watch?v=c3mPdZA-Fmc&list=PL5Q2soXY2Zi9xidyIgBxUz7xRPS-wisBN + +# Maths + +- https://math.stackexchange.com/questions/1652937/a-good-book-for-beginning-group-theory + +# Miscellaneous + +- https://www.dabeaz.com/coroutines/ + +- https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html + +- https://henrikwarne.com/2024/02/11/finding-a-new-software-developer-job/ + +- https://gist.github.com/cb372/5f6bf16ca0682541260ae52fc11ea3bb diff --git a/c++.md b/c++.md new file mode 100644 index 0000000..d89428a --- /dev/null +++ b/c++.md @@ -0,0 +1,38 @@ +--- +title: C++ +--- + +# C++ + +## Why `operator[]` returns a reference + +```c++ +class IntList +{ + private: + int m_list[10]{}; + + public: + int& operator[] (int index); +}; + +int& IntList::operator[] (int index) +{ + return m_list[index]; +} +``` + +Let’s take a closer look at how `list[2] = 3` evaluates. Because the subscript operator has a higher precedence than the assignment operator, `list[2]` evaluates first. `list[2]` calls `operator[]`, which is defined to return a reference to `list.m_list[2]`. Because `operator[]` is returning a reference, it returns the actual `list.m_list[2]` array element. The partially evaluated expression becomes +`list.m_list[2] = 3`, which is a straightforward integer assignment. + +Any value on the left hand side of an assignment statement must be an l-value (which is a variable that has an actual memory address). Because the result of `operator[]` can be used on the left hand side of an assignment (for example `list[2] = 3`), the return value of `operator[]` must be an l-value. As it turns out, references are always l-values, because you can only take a reference of variables that have memory addresses. So by returning a reference, the compiler is satisfied returning an l-value. + +Consider what would happen if `operator[]` returned an integer by value instead of by reference. `list[2]` would call `operator[]`, which would return the value of `list.m_list[2]`. For example, if `m_list[2]` had the value of 6, `operator[]` would return the value 6. `list[2] = 3` would partially evaluate to `6 = 3`, which makes no sense. If you try to do this, the C++ compiler complains: + + C:VCProjectsTest.cpp(386) : error C2106: '=' : left operand must be l-value + +Taken from [Overloading the Subscript Operator](https://www.learncpp.com/cpp-tutorial/overloading-the-subscript-operator/). + +### Necessity of a virtual destructor for an abstract class + +A virtual destructor is essential for an abstract class because an object of a derived class is usually manipulated through the interface provided by its abstract base class. In particular it may be deleted through a pointer to a base class. Then the virtual function call mechanism ensures that the proper destructor is called. That destructor then implicitly invokes the destructors of its bases and members. diff --git a/fpga.md b/fpga.md new file mode 100644 index 0000000..c59bac6 --- /dev/null +++ b/fpga.md @@ -0,0 +1,7 @@ +--- +title: FPGA +--- + +# Resources + +- https://projectf.io/sitemap/ diff --git a/gpg-key-migration.md b/gpg-key-migration.md new file mode 100644 index 0000000..b5a0756 --- /dev/null +++ b/gpg-key-migration.md @@ -0,0 +1,62 @@ +# Migrate gpg keys from one workstation to another + +Replace **[your key]** with your key ID + +To obtain your key ID +```bash +gpg --list-secret-keys --keyid-format LONG +``` + +Which returns something like +```bash +/home/angela/.gnupg/pubring.kbx +------------------------------- +sec rsa4096/[your key] 2018-03-30 [SC] + ABCDEFGHIJKLMNOPQRSTUVWXYZ +uid [ unknown] angela (KEY NAME) +ssb rsa4096/ABCDEFGHIJKL 2018-03-30 [E] + +``` + +After the key size `rsa4096/` is your key ID. + +*** + +Export the key in preparation to move it +```bash +gpg --export -a [your key] > gpg-pub.asc +``` + +Prepare the secret key for migration (if password protected, you'll be prompted to enter it) +```bash +gpg --export-secret-keys -a [your key] > gpg-sc.asc +``` + +Find them +```bash +ls -l gpg*.asc +``` + +Drag the key pair from the current directory to your USB stick or however else you move them. + + +Once on the new machine, import them +```bash +gpg --import gpg-pub.asc +``` + +If password protected, you'll be prompted to enter it +```bash +gpg --import gpg-sc.asc +``` + +If you need to adjust the trust level +```bash +gpg --edit-key [your key] +``` + +One thing to do was restart gpg-agent after importing the public key and adjusting the trust level. The secret key couldn't be imported till restarting the gpg agent. + +```bash +sudo gpgconf --kill gpg-agent +``` diff --git a/gstreamer/gstreamer.md b/gstreamer/gstreamer.md new file mode 100644 index 0000000..b255506 --- /dev/null +++ b/gstreamer/gstreamer.md @@ -0,0 +1,256 @@ +--- +title: GStreamer +--- + +# Internals + +## RTP timestamp + +- Domain for `rtptime` is `clock-time` from SDP +- 48000 in this case +- So one tick is 1/48000 of a second +- Meaning 20 milliseconds is 960 + +## Discussion on threading internals + + + +## Threads and probes + +[Probes](https://gstreamer.freedesktop.org/documentation/additional/design/probes.html?gi-language=c) + +[Threads](https://gstreamer.freedesktop.org/documentation/application-development/advanced/threads.html?gi-language=c) + +## Audio decoders + +There is `max-errors` property on audio decoder elements which can be helpful in debugging. + +## Autoplugging + +- [Autoplugging](https://gstreamer.freedesktop.org/documentation/application-development/advanced/autoplugging.html?gi-language=c) + +- [Playback Components](https://gstreamer.freedesktop.org/documentation/application-development/highlevel/playback-components.html?gi-language=c) + +- [Decodebin Design](https://gstreamer.freedesktop.org/documentation/additional/design/decodebin.html?gi-language=c) + +## Why are queues required + +- [Demuxing mp4 and queues](http://gstreamer-devel.966125.n4.nabble.com/demuxing-mp4-and-queues-td4689905.html) + +This is a classical problem, and a good illustration why you usually need queues after elements with N src pads, and before elements with N sink pads. + +The demuxer is pushing to its source pads from the same thread, and if you have connected these source pads with sync=true sinks, their chain function blocks until prerolling is done, that is until each sink with sync=true has received a buffer. Adding queues decouples branches of the pipeline, as they will start their own streaming thread. + +## Dynamically remove or unlink elements + +- http://gstreamer-devel.966125.n4.nabble.com/Dynamically-adding-and-removing-branches-of-a-tee-td973635.html + +- https://gstreamer.freedesktop.org/documentation/application-development/advanced/pipeline-manipulation.html?gi-language=c + +See probes here. + +## Webkit debugging + +- https://eocanha.org/blog/2021/05/25/gstreamer-webkit-debugging-by-using-external-tools-2-2/ + +## Quark + +Quarks represent a different idea Quarks are basically an internal "registry" of strings. The idea is that if you want to compare strings often, that can be inefficient, so you make a GQuark of it (essentially a hash) that you can stably compare. So if there is a string comparison you are performing often, or you want to make faster, you convert the string to a quark (hashing it to an integer), and then use that instead. + +## Flushing probe + +https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/274 + +# Elements + +## GstDiscoverer + +- Internally uses `uridecodebin`. This element decodes data from a URI into raw media. It selects a source element that can handle the given URI scheme and connects it to a `decodebin` element. It acts like a demuxer, so it offers as many source pads as streams are found in the media. + + ``` bash + gst-launch-1.0 uridecodebin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! videoconvert ! autovideosink + ``` + +- Primary function seems to be `discoverer_collect` which gets called pipeline is pre-rolled + +- The next important function is `parse_stream_topology`. It can also be called recursively + + - Checks for `next` in `GstStructure` passed as argument to this function + + - If `next` is `NULL`, then it's the end of the stream, else one has a `GstValue` holding either a structure or a list + + - For the case of a `GstStructure`, `collect_information` is then called. It reads various fields in the caps to collect information like `rate`, `channels` and `channel-mask`. Something called `TAGS` and `QUARK` also come into picture here. Quarks seem to be a 2 way association between a string and a unique identifier + + - For the case of list, along with some things, walk over the list, calling `parse_stream_topology` for each item in the list, where the item seems to a sub stream + +### Decodebin + +This element automatically constructs a decoding pipeline using available decoders and demuxers via auto-plugging until raw media is obtained. It is used internally by `uridecodebin` which is often more convenient to use, as it creates a suitable source element as well. It replaces the old `decodebin` element. It acts like a demuxer, so it offers as many source pads as streams are found in the media. + + ``` bash + gst-launch-1.0 souphttpsrc location=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm ! decodebin ! autovideosink + ``` + +### Thumbnails + +Can `GstDiscoverer` be used for thumbnails? Googling a bit, and we land on this issue [discoverer should generated video/image thumbnails](https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/58). This references [Thumbnailer Spec](https://specifications.freedesktop.org/thumbnail-spec/thumbnail-spec-latest.html) and [Thumbnail Management DBus Specification](https://wiki.gnome.org/action/show/DraftSpecs/ThumbnailerSpec?action=show&redirect=ThumbnailerSpec). `GstDiscoverer` does not seem to have a thumbnailing API. + +There is a totem video thumbnailer which is the video thumbnailer for the **GNOME** desktop but seems not to be available as a separate library. [ffmpegthumbnailer](https://github.com/dirkvdb/ffmpegthumbnailer) based on ffmpeg is a lightweight video thumbnailer that can be used by file managers to create thumbnails for video files. However, packaging `ffmpeg` might be a concern here and `ffmpegthumbnailer` has a *GPL-2.0* license. FFmpeg is licensed under the LGPL license, however, if a particular build of FFmpeg is linked against any GPL libraries (notably x264), then the entire binary is licensed under the GPL. + +If this needs to be provided with GStreamer then `gstbasesink` has a note on `last-sample` property where it mentions that the property can be used to generate thumbnails. Also see `gst_base_sink_get_last_sample`. A related option seems to be `convertframe` which has two APIs for converting a raw video buffer into the specified output caps. See [convertframe](https://gstreamer.freedesktop.org/documentation/video/convertframe.html?gi-language=c#). + +Some links which have examples. + +- https://github.com/GStreamer/gst-editing-services/blob/master/examples/c/thumbnails.c +- https://stackoverflow.com/questions/15789652/how-to-create-video-thumbnails-with-python-and-gstreamer +- https://gist.github.com/dplanella/5563018 + +### Tracker miner + +`GstDiscoverer` is used in Tracker Miner though not for everything. From [Into the Pyramid](https://samthursfield.wordpress.com/2019/12/01/into-the-pyramid/). + +- Tracker has a ‘generic image extraction’ rule that tries to find metadata for any image/\* MIME type that isn’t a .bmp, .jpg, .gif, or .png. This codepath uses the GstDiscoverer API, the same as for video and audio files, in the hope that a GStreamer plugin on the system can give us useful info about the image. + +- The GstDiscoverer instance is created with a timeout of 5 seconds. (This seems quite high — the gst-typefind utility that ships with GStreamer uses a timeout of 1 second). + +- GStreamer’s GstDiscoverer API feeds any file where the type is unknown into an MPEG decoder, which is effectively an unwanted fuzz test and can trigger periods of high CPU and memory usage. + +- 5 seconds of processing non-MPEG data with an MPEG decoder is somehow enough to cause Linux’s scheduler to lock up the entire system. + +## Rank + +[Modify the elements rank](https://developer.ridgerun.com/wiki/index.php?title=GStreamer_modify_the_elements_rank) + +Each GStreamer element has a detail called rank that defines the priority used by the autoplugger when it wants to connect a new element but has multiple options. + +# Pipeline examples + +## Opus RTP encoding decoding pipeline + +Note that in place of rtpjitterbuffer, rtpbin also works. + +``` bash +export AUDIO_CAPS="application/x-rtp,media=(string)audio,clock-rate=(int)48000,encoding-name=(string)OPUS" +gst-launch-1.0 -v audiotestsrc ! audioconvert ! audioresample ! opusenc ! rtpopuspay ! udpsink host=localhost port=50004 +gst-launch-1.0 udpsrc caps=$AUDIO_CAPS address=localhost port=50004 ! rtpjitterbuffer latency=20 do-lost=true ! rtpopusdepay ! opusdec plc=true ! pulsesink +``` + +## Tearing down a branch of *tee* + +``` python +tee = pipe.get_by_name(RTMP_TEE) +srcPad = tee.get_static_pad("src_" + str(idx)) +srcPad.add_probe( + Gst.PadProbeType.BLOCK, rtmp_tee_srcpad_block_probe, rtmpUrl +) + +def rtmp_tee_srcpad_block_probe( + pad: Gst.Pad, info: Gst.PadProbeInfo, rtmpUrl +) -> Gst.PadProbeReturn: + tee = pipe.get_by_name(RTMP_TEE) + queue = pipe.get_by_name(MUX_OUTPUT_QUEUE + "-" + rtmpUrl) + rtmpSink = pipe.get_by_name(RTMP_SINK + "-" + rtmpUrl) + sinkPad = queue.get_static_pad("sink") + pad.unlink(sinkPad) + rtmpSink.set_state(Gst.State.NULL) + queue.set_state(Gst.State.NULL) + tee.remove_pad(pad) + pipe.remove(queue) + pipe.remove(rtmpSink) + return Gst.PadProbeReturn.REMOVE +``` + +## Check camera capabilities + +``` bash +gst-device-monitor-1.0 Video/Source +v4l2-ctl --list-formats-ext -d0 +``` + +# Annotations + +[Annotation Glossary](https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html//annotation-glossary.html) + +# AAC test samples + +- https://samples.mplayerhq.hu/A-codecs/AAC/ + +# Debugger + +If `SIGSEGV` occurs, to use `gdb` and `gst-launch` to debug the crashed process, first run + +``` bash +echo 0 > /proc/sys/kernel/yama/ptrace_scope +echo -1 > /proc/sys/kernel/perf_event_paranoid +``` + +- Arch enables the Yama LSM by default, which provides a `kernel.yama.ptrace_scope` kernel parameter. This parameter is set to 1 (restricted) by default which prevents tracers from performing a ptrace call on traces outside of a restricted scope unless the tracer is privileged or has the `CAP_SYS_PTRACE` capability. + + +# Terminology + +- **Transmuxing**, also referred to as repackaging or packetizing, is a process in which audio and video files are repackaged into different delivery formats without changing the files' contents. + +# Yocto + +[Enable RTMP plugin](https://www.toradex.cn/community/questions/13303/gstreamer-rtmp-plugin.html) + +# RTP + +[Some FAQs about RTP](https://www.cs.columbia.edu/~hgs/rtp/faq.html) + +# Basics of time and synchronization + +- https://www.youtube.com/watch?v=EOHl1ktWwp4 + +- https://www.youtube.com/watch?v=OKTfNAmFTo4 + +- `running_time` is the amount of time "spent" in playing + +- Playing => Paused: Remember running time + +- Paused => Playing: base_time = current absolutetime - running_time + +# Live pipeline + +## Live sources + +- https://gstreamer.freedesktop.org/documentation/additional/design/live-source.html?gi-language=c + +- https://gstreamer.freedesktop.org/documentation/additional/design/element-source.html?gi-language=c + +-https://gstreamer.freedesktop.org/documentation/base/gstbasesrc.html?gi-language=c + +## Pipeline to check base time gets updated after PAUSE -> PLAY transition + +``` bash +env GST_DEBUG=basesink:6 gst-play-1.0 /usr/share/sounds/alsa/Front_Left.wav --audiosink="fakesink sync=true" -q 2>&1 | grep base_time +``` + +# Running test suite with gdb + +``` bash +ninja -C build && env CK_FORK=no meson test --gdb -C build --suite gst-plugins-good elements_splitmuxsink +``` + +# Re-timestamping frames + +- https://kaustav.space/2018-09-20/retimestamping-frames-in-gstreamer + +# H264 + +- https://doc-kurento.readthedocs.io/en/latest/knowledge/h264.html + +# Video mp4 + +- https://bitmovin.com/container-formats-fun-1 +- https://docs.unified-streaming.com/documentation/live/subtitles.html#embedded-subtitles-captions +- https://stackoverflow.com/questions/24884827/possible-locations-for-sequence-picture-parameter-sets-for-h-264-stream +- https://stackoverflow.com/questions/10185474/gstreamer-how-to-get-audio-and-video-to-play-at-the-same-rate +- https://bitmovin.com/fun-with-container-formats-2/ +- https://www.ramugedia.com/mp4-container +- https://gpac.github.io/mp4box.js/test/filereader.html +- https://en.wikipedia.org/wiki/Video_compression_picture_types +- https://en.wikipedia.org/wiki/Group_of_pictures +- https://observablehq.com/@benjamintoofer/iso-base-media-file-format +- https://streaminglearningcenter.com/blogs/open-and-closed-gops-all-you-need-to-know.html diff --git a/haskell/haskell.md b/haskell/haskell.md new file mode 100644 index 0000000..b7e5df2 --- /dev/null +++ b/haskell/haskell.md @@ -0,0 +1,347 @@ +--- +title: Haskell +--- + +# Haskell + +## Semirings + +- https://doisinkidney.com/posts/2016-11-17-semirings-lhs.html + +- https://doisinkidney.com/posts/2017-10-13-convolutions-and-semirings.html + +- https://r6.ca/blog/20110808T035622Z.html + +## Monoids + +- http://blog.sigfpe.com/2009/01/haskell-monoids-and-their-uses.html + +- https://www.haskellforall.com/2014/07/equational-reasoning-at-scale.html + +- https://apfelmus.nfshost.com/articles/monoid-fingertree.html + +- https://gist.github.com/cscalfani/b0a263cf1d33d5d75ca746d81dac95c5 + +## Continuation passing style + +- https://stackoverflow.com/questions/45334985/when-to-use-cps-vs-codensity-vs-reflection-without-remorse-in-haskell + +- https://deque.blog/2017/01/13/beautiful-code-control-fold/ + +- https://free.cofree.io/2020/01/02/cps/ + +- https://neilmitchell.blogspot.com/2014/06/optimisation-with-continuations.html + +- https://jtobin.io/page2/ + +## How to read sequential calculus notation + +![Sequential Calculus Notation Helper](../images/sequent-calculus.jpg "SequentCaculusHelper") + +- http://logitext.mit.edu/tutorial + +- https://langdev.stackexchange.com/questions/2692/how-should-i-read-type-system-notation + +## Folds + +Alexis King's excellent explanation on folds + +- https://twitter.com/lexi_lambda/status/1198676516445589509 + +- https://github.com/hasura/graphql-engine/pull/2933#discussion_r328821960 + + +## Principles of Programming Languages + +- http://kmicinski.com//cis352-s20/ + +- https://www.youtube.com/user/krismicinski/videos + +# Category Theory + +## Resources + +- https://pages.cs.wisc.edu/~jcyphert/categoryTheoryNotes/ +- http://brendanfong.com/programmingcats.html +- https://www.johndcook.com/blog/2018/08/06/hom-functors/ +- https://www.cs.cornell.edu/courses/cs6117/2022fa/ + +## Chapter 1 + +- A category consists of objects and arrows that go between them. +- The essence of a category is composition or the essence of composition is a category. + +### Arrows as functions + +- Arrows are also called morphisms +- In a category, if there is an arrow going from **A** to **B** and an arrow going from **B** to **C** then there must also be a direct arrow from **A** to **C** that is their composition. Note that this is not a full category because it's missing identity morphisms. +- **In Mathematics and in Haskell, functions compose right to left**. + +### Properties of Composition + +- Composition is associative. +- For every object, there is identity morphism. + +### Notes from video lecture "1.2 What's a category?" + +Major tools in arsenal + +- Abstraction +- Composition +- Identity + +*Composition and Identity define category theory*. + +Category is just a bunch of objects. One analogy is thinking of category as a graph. + +Composable "end of one is beginning of other". And remember that end or beginnings are objects. Thinking in terms of types, the objects should be types?. If the *end* type does match *beginning* type, we cannot compose right? + +When we are defining a category, the category is defined by giving objects, what the objects are, defining arrows and then defining composition. Composition is like a big multiplication table for arrows. For every arrow, one has to define their composition. + +Different composition tables give different category. + +We also have laws: + +1. Left Identity +2. Right Identity +3. Associativity *h* ∘ (*g*∘*f*) = (*h*∘*g*) ∘ *f* + +Objects don't form a set in general. If they form a set, then this category is called *small*. Morphisms between any two objects form a set. There is a category theory in which they don't form a set and that +is a higher order categories. + +A group is an monoid that also has an inverse. A groupoid is a category in which every arrow has an inverse. + +Objects are types and arrows are functions. A function is a morphism between two types. + +Question: + +1. hom-sets map pairs to sets. And sets are objects in the category set. How does it follow we have a mapping between categories. + +## Chapter 2 + +### Types are about composability + +### What are types? + +- Types are sets of values. +- In the ideal world, Haskell types are sets and Haskell functions are mathematical functions between sets. +- Functions that return bottom are called partial, as opposed to total functions, which return valid results for every possible argument. +- Because of bottom, the category of Haskell types and functions are referred to as **Hask** rather than **Set**. + +### Why do we need a mathematical model? + +- Operational semantics describes the mechanics of program execution. +- In denotational semantics, every programming construct is given it's mathematical interpretation. + +### Examples of types + +- What's the type corresponding to an empty set? This type is called **Void** in Haskell. +- Functions that can be implemented with the same formula for any type are called parametrically polymorphic. + +### Notes from video lecture "2.1 Functions, Epimorphisms" + +- Category **Set** which contains sets and functions between sets. +- The set of all paris forms a Cartesian Product. A subset of the Cartesian Product is a **relation**. +- If a function does not collapse things, it is *injective*. +- If the function covers the whole co-domain, then it is *surjective*. +- Both *injective* and *surjective*, then it is an isomorphism. +- Think of Epimorphism as a way of considering surjection in Category theory. Since we do not care about elements, but, need to think in terms of morphisms, this is where Epimorphism comes in. Epimorphisms are categorical analogues of surjective functions. +- Surjective => Epic, Injective => Monic + +### Notes from video lecture "2.2 Monomorphisms, simple types" + +Equality of composition leads to Equality of functions. + +*Void* in logic corresponds to *False*. + +## Chapter 3 + +### Simple Graphs + +Creating a category, which has an object for every node in the graph, and all possible chains of composable graph edges as morphisms. One may even consider identity morphisms as special cases of chains of length zero. + +Such a category is called *free category* generated by a given graph. It's an example of free construction, a process of completing a given structure by extending it with a minimum number of items to satisfy it's laws (here, the laws of a category). + +### Orders + +Consider the category where a morphism is a relation between objects: the relation of being less than or equal. We have identity morphisms as every object is less than or equal to itself. If `a <= b` and `b <= c` then `a <= c`. Composition is also associative. A set with a relation like this is called *preorder*, so a preorder is indeed a category. + +For a stronger relation: + +- An additional condition that, if `a <= b` and `b <= a`, then *a* miust be the same as *b*. That's called a *partial order*. +- If any two objects are in relation with one each other, one way or another, that gives a *linear order* or *total order*. + +A preorder is a category where there is at most one morphism going from any object *a* to any object *b*. Another name for such a category is "thin". A preorder is a thin category. + +A set of morphisms from object *a* to object *b* in a category **C** is called a *hom-set* and is written as **C**(*a*,*b*) or sometimes **Hom***C*(*a*,*b*). *Hom* stands for homomorphism, the usual name for structure preserving functions in algebra. + +Sorting algorithms can only work on total orders. Partial orders can be sorted using topological sort. + +### Monoid as Set + +A monoid is defined as a set with binary operation. The operation should be associative and should have one special element that behaves likes a unit with respect to it. + +``` haskell +class Monoid m where + mempty :: m + mappend :: m -> m -> m +``` + +Interpretation of the type signature of *mappend* as *m* → (*m*→*m*) tells us that *mappend* maps an element of a monoid set to a function acting on that set. + +There is the adders example and then this follows. What does this mean? As in I haven't digested it! + +One might ask the question whether every categorical monoid - a one object category - defines a unique set-with-binary-operator monoid. It turns out we can always extract a set from a single-object category. This is set of morphisms - the adders in our example. In other words, we have the *hom-set*, **M**(*m*,*m*) of the single object *m* in the category **M**. + +The monoidal product of two set-elements is the element corresponding to the composition of the corresponding morphisms. Given two elements of **M**(*m*,*m*) corresponding to *f* and *g*, their product will correspond to the composition of *f* ∘ *g*. The composition always exists, because the source and the target for these morphisms are the same object. And it's associative by the rules of the category. The identity morphism is the neutral element of this product. So we can always recover a set monoid from a category monoid. For all intents and purposes they are one and the same. + +A category in which morphisms between any two objects form a set is called locally small. + +### Notes from video lecture "3.1 Examples of categories, orders, monoids" + +The set of arrows between any two objects is called a *hom-set*. + +## Chapter 4 + +A Kleisli category has objects, the types of underlying programming language. Morphisms from type **A** to type **B** are functions that go from **A** to a type derived from **B** using the particular embellishment. Each Kleisli category defines its own way of composing such morphisms, as well as the identity morphisms with respect to that composition. We will learn that imprecise term *embellishment* corresponds to the notion of an endofunctor in a category. + +### Notes from video lecture "3.2 Kleisli Category" + +Every set is a subset of itself. In terms of order, we call it reflexivity. How would we translate composition into this language of being a subset? If *A* is a subset of *B* and *B* is a subset of *C*, *A* is a subset of *C*. It's a partial order, there are no loops as *A* ⊆ *B* and *B* ⊆ *A* then *A* = *B*. + +Monad is nothing special. It is just a way of composing functions. + +## Chapter 5 - Products and Coproducts + +Common construction in category theory called the *universal construction* for defining objects in terms of their relationships. + +### Initial Object + +The *initial object* is the object that has one and only one morphism going to any object in the category. The uniqueness of the initial object is not guaranteed. Only *uniqueness up to isomorphism*. + +The initial object in a partially ordered set (often called a poset) is its least element. In the category of sets and functions, the initial object is the empty set. + +### Terminal Object + +The **terminal object** is the object with one and only one morphism coming to it from any object in the category. The terminal is unique upto isomorphism. + +In a poset, the terminal object if it exists, is the biggest object. In the category of sets, the terminal object is a singleton. Corresponds to the unit type () in Haskell. + +### Notes from video lecture "4.1 Terminal and initial objects" + +Singleton set has an arrow coming from every other set. There is a function from any type to unit (). + +∀*a*∃*f* :  : *a* → () + +For all objects (type) *a*, there exists an *f*, that goes from *a* to +the terminal object (). + +∀*f* :  : *a* → (), *g* :  : *a* → () ⟹ *f* ≡ *g* + +For every two functions, from *a* to the terminal object, if we have *f* and *g*, also from *a* to the terminal object, then *f* ≡ *g*. + +### Notes from video lecture "4.2 Products" + +Outgoing arrows from the terminal object? There are outgoing arrows from a terminal object. These arrows help us define the generalised elements in other objects. Every arrow from a terminal object to another object, is a definition of a generalised element in this other object. + +### Notes from video lecture "5.1 Coproducts, Sum Types" + +Discriminated Union/Tagged Union/Variant. + +### Duality + +The only difference between the above two was the direction of morphisms. For any category **C** we can define the *opposite category* **C**op just by reversing all the arrows. The opposite category automatically satisfies all the requirements of a category, as long as we simultaneously redefine composition. + +The set of arrows between any two objects is called a *hom-set*. + +If original morphisms *f* :: *a* → *b* and *g* :: *b* → *c* composed to *h* :  : *a* → *c* with *h* = *g* ∘ *f*, then the reversed morphisms + +*f*op :  : *b* → *a* and +*g*op :  : *c* → *b* will compose to +*h*op :  : *c* → *a* with +*h*op = *f*op ∘ *g*op + +Reversing the identity arrows is a no-op. + +### Isomorphisms + +Isomorphic objects look the same - they have the same shape. It means that every part of one object corresponds to some part of another object in a one-to-one mapping. + +Mathematically it means that there is a mapping from object *a* to object *b*, and there is a mapping from object *b* to object *a*, and they are the inverse of each other. + +In category theory, we replace mappings with morphisms. An isomorphism is an invertible morphism, or a pair of morphisms, one being the inverse of the other. + +Morphism *g* is the inverse of morphism *f* if their composition is the identity morphism. + +``` haskell +f . g = id +g . f = id +``` + +### Products + +A **product** of two objects *a* and *b* is the object *c* equipped with two projections such that for any other object *c*′ equipped with two projections there is a unique morphism *m* from *c\`* to *c* that factorizes these projections. + +A **coproduct** of two objects *a* and *b* is the object *c* equipped with two injections such that for any other object *c\`* equipped with two injections there is a unique morphism *m* from *c* to *c\`* that factorizes those injections. + +### Asymmetry + +Product behaves like multiplication, with the terminal object playing the role of one; whereas coproduct behaves more like the sum, with the initial object playing the role of zero. In particular for finite sets, the size of the product is the product of the sizes of the individual sets, and the size of the coproduct is the sum of the sizes. + +## Chapter 6 + +Set (the category of sets) is a *monoidal category*. It's a category that is also a monoid, in the sense that one can multiply objects (take their Cartesian Product). + +A product of two types *a* and *b* must contain both a value of type *a* and a value of type *b*, which means both types must be inhabited. A sum of two types on the other hand, contains either a value of type *a* or a value of type *b*, so it's enough if one of them is inhabited. Logical *and* and *or* also form a semiring, and it too can be mapped into type theory. + +## Chapter 7 + +Functor composition is associative (the mapping of objects is associative, and the mapping of morphisms is associative). There is also a trivial identity functor in every category: it maps every object to itself and every morphism to itself. So functors have all the same properties as morphisms in some category. It would have to be a category in which objects are categories and morphisms are functors. It's a category of categories. + +### Notes from video lecture "6.1 Functors" + +If the mapping of *hom-sets* is injective, then we call such functors *faithful*. A faithful functor is injective on all *hom-sets*. If it is *ful*, then it is surjective. + +## Chapter 8 + +A monoidal category defines a binary operator acting on objects, together with a unit object. **Set** is a monoidal category with respect to Cartesian product, with the singleton set as a unit. It's also a monoidal category with respect to disjoint union. One of the requirements for a monoidal category is that the binary operator should be a bifunctor. This is an important requirement, as the monoidal product should be compatible with the structure of the category, which is defined by morphisms. + +### Notes from video lecture "7.1 Functoriality, Bifunctors" + +Functors are mappings between categories. Endofunctors are mapping within a category. + +"Basically, when you have some expression E that contains F, and we say E is functorial in F, it means that F → E, is a functor." + +- https://math.stackexchange.com/questions/2009361/what-does-it-mean-to-be-functorial-in-something + +## Chapter 9 + +### Notes from video lecture "8.1 Function Objects, Exponentials" + +## Chapter 10 + +A parametrically polymorphic function between two functors (including the edge case `Const` functor) is always a natural transformation. Since all standard algebraic data types are functors, any polymorphic function between such types is a natural transformation. + +### Notes from video lecture "9.1 Natural Transformations" + +- Category is about structure. +- Functors are mappings between categories that preserve structure. They take a category and embed it in another category. It's like searching for a pattern inside a category or modelling a category inside another category. +- Categories and functors are necessary to define natural transformations. + +## Chapter 12 + +- https://www.math3ma.com/blog/limits-and-colimits-part-1 +- https://www.math3ma.com/blog/limits-and-colimits-part-2 +- https://www.math3ma.com/blog/limits-and-colimits-part-3 +- https://kavigupta.org/2016/05/10/A-Haskell-Class-For-Limits/ +- http://blog.sigfpe.com/2010/03/products-limits-and-parametric.html + +One interpretation of limits and co-limits with respect to Haskell is how they play with universal and existential quantification. We can write + +``` haskell +{-#Language ExistentialQuantification#-} +newtype Limit f = Limit (forall a. f a) +data Colimit f = forall a. Colimit ( f a) +``` + +So *Limit* universally quantifies over all types a functor *f* can map over, and *CoLimit* let's us talk about an *f* *a* but forget about the exact *a* (existential quantification). diff --git a/images/sequent-calculus.jpg b/images/sequent-calculus.jpg new file mode 100644 index 0000000..91220ec Binary files /dev/null and b/images/sequent-calculus.jpg differ diff --git a/notes.md b/notes.md new file mode 100644 index 0000000..816f4c7 --- /dev/null +++ b/notes.md @@ -0,0 +1,172 @@ +--- +title: Notes +--- + +# Git + +## Rebase + +The use of -Xtheirs and -Xours appear to be somewhat counterintuitive, so think of it as telling git which branch code to favor when resolving rebase conflicts. For example, when doing: + +``` bash +git rebase -Xtheirs branch-b +``` + +Xtheirs favors your current branch-a code when overwriting merge conflicts, and vice versa -Xours overwrites merge conflicts with the code in branch-b. + +Similar options exist in git merge command as well, but the meaning of -Xtheirs and -Xours is reversed due to the differences on how git rebase and git merge operate and what they consider ours vs. theirs: + +``` bash +git rebase -Xtheirs branch-b # <- ours: branch-b, theirs: branch-a +git merge -Xtheirs branch-b # <- ours: branch-a, theirs: branch-b +``` + +Thus, if you are merging changes from origin/master and would like git to favor your current branch code during merge conflicts, you’d need to do this: + +``` bash +git merge -Xours origin/master +``` + +- https://nitaym.github.io/ourstheirs/ + +- https://dev.to/cloudx/resolving-git-conflicts-in-package-lockjson-using-a-merge-driver-2ipk + +- https://ten0s.github.io/blog/2020/04/19/git-difftool-and-binary-files + +- https://gist.github.com/pascaliske/53648334378d592c0c2cc62b989a027e + +# Shell tips + +## Recursively remove execute permissions + +``` bash +chmod -R a-x+X,u-x+rwX,go-wx+rX Study/ +``` + +## SED debugger - sedsed + +- https://aurelio.net/projects/sedsed/ + +# System related + +## Htop explained + +- https://peteris.rocks/blog/htop/ + +## Using perf + +- `perf top -t ` shows the performance counters + +## rsync + +``` bash +rsync -a username@remote_host:/home/username/dir1 place_to_sync_on_local_machine +``` + +Like ~cp~ and similar tools, the source is always the first argument and the destination is always the second. + +## Nginx hosting setup + +- [Understanding DNS record types](https://www.name.com/support/articles/205516858-Understanding-DNS-record-types) + +- [linuxize: Secure Nginx with letsencrypt on Ubuntu 20.04](https://linuxize.com/post/secure-nginx-with-let-s-encrypt-on-ubuntu-20-04/) + +- [DigitalOcean: Secure Nginx with letsencrypt on Ubuntu 20.04](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04) + +- [Self hosting git](https://peppe.rs/posts/self-hosting_git/) + +The below is used to generate the certificates. To add more sub-domains, just call the command again, adding the sub domain. + +```bash +certbot certonly --agree-tos --email sanchayan@sanchayanmaity.net --webroot -w /var/lib/letsencrypt/ -d sanchayanmaity.net -d www.sanchayanmaity.net -d git.sanchayanmaity.net -d monitor.sanchayanmaity.net +certbot certonly --agree-tos --email sanchayan@sanchayanmaity.com --webroot -w /var/lib/letsencrypt/ -d sanchayanmaity.com -d www.sanchayanmaity.com -d git.sanchayanmaity.com -d monitor.sanchayanmaity.com +``` + +- [Serving static content](https://docs.nginx.com/nginx/admin-guide/web-server/serving-static-content/) + +## Setting up a kernel debugging environment + +- https://pwning.systems/posts/setting-up-a-kernel-debugging-environment/ + +## Javascript + +- https://javascript.info/ + +- http://superherojs.com/ + +# C + +[Caching integer overflows in C](https://www.fefe.de/intof.html) + +# Where to find good bread in Bangalore + +- http://www.thevinebangalore.com/where-to-get-good-bread-in-bangalore/ + +# Wayland + +[A better way to read Wayland documentation](https://wayland.app/protocols/) + +# Distributed systems + +PI calculus + +# Yocto + +Yocto/openembedded builds never work on Arch and even on something old like Ubuntu 16.04, one runs into problems like this + +- https://git.yoctoproject.org/cgit/cgit.cgi/poky/patch/meta/recipes-devtools/m4?id=95ca077ab871ceff46c2052f324f879a1d624ff4. + +docker is a blessing for such things. + +``` bash +docker run --rm -it -v {$PWD}:/oe-core crops/poky:debian-9 --workdir=/oe-core +``` + +# Pulseaudio + +- https://endless.ersoft.org/pulseaudio-loopback/ + +# Keyboard websites for split ergonomic keyboards + +- *ergomech.store* + +- *keebmaker.com* + +- *boardsource.xyz* + +- *littlekeyboards.com* + +- *splitkb.com* + +- *rectangles.store* + +- *bastardkb.com* + +- *keeb.io* + +- + +- *stackskb.com* + +# Fish + +- https://github.com/jorgebucaran/cookbook.fish#how-do-i-redirect-stdout-or-stderr-to-a-file-in-fish +- https://developerlife.com/2021/01/19/fish-scripting-manual/#how-to-write-functions + +# GPG + +- https://emailselfdefense.fsf.org/en/index.html +- https://oguya.ch/posts/2016-04-01-gpg-subkeys/ +- https://gitlab.com/muttmua/mutt/-/wikis/MuttGuide/UseGPG +- https://riseup.net/en/security/message-security/openpgp/best-practices +- https://m7i.org/tips/mutt-pgp-sign-attachment/ +- https://thoughtbot.com/blog/pgp-and-you +- https://code.dblock.org/2021/04/16/adding-work-email-to-a-gpg-key-and-signing-git-commits.html diff --git a/pulseaudio.md b/pulseaudio.md new file mode 100644 index 0000000..30b59d3 --- /dev/null +++ b/pulseaudio.md @@ -0,0 +1,83 @@ +--- +title: Projects +--- + +### AAC observations + +The following are the observations from the Wireshark capture traces regarding the issue observed while using AAC with SRS-XB33. + +1. Android and SRS-XB33 + + - Remote sends `GetAllCapabilities` command and gets `SBC`, `MPEG-2/4 AAC` and `LDAC` as supported in the response. + + - In case of `AAC`, the following flags are set on Android side + + - MPEG-2 AAC LC + - Sampling rate of 44.1 KHz + - VBR supported + - 2 channels + + - The `SetConfiguration` gets send with the following parameters set + + - MPEG-2 AAC LC + - Sampling rate of 44.1 KHz + - VBR supported + - 2 channels + +2. PulseAudio and SRS-XB33 + + 1. Initial Connection + + - The device sends a `Discover` request, followed by `GetAllCapabilities` and then `SetConfiguration`. + + - For the `GetAllCapabilities`, `MPEG-2,4 AAC` and `SBC` are returned by bluez. + + - The `GetAllCapabilities` command, seems to be send twice from the device and in both cases return the exact same capabilities. This happens for both `AAC` and `SBC`. + + - The `SetConfiguration` command send is send by SRS-XB33 and has the following parameters set + + - MPEG-4 AAC LC + - Sampling rate of 48 KHz + - 2 channels + - VBR not supported + + 2. Reconnection + + In the reconnection case, the `SetConfiguration` command has the following parameters set. + + - Sampling rate of 44.1 KHz + - MPEG-4 AAC LC + - 2 channels + - VBR not supported + +### Codec switching + +- Currently the following profiles exist + + 1. headsetaudiogateway + 2. headsetheadunit + 3. a2dpsource + 4. a2dpsink + +- For backchannel, two additional profiles are needed + + 1. a2dpsourcewithbackchannel + 2. a2dpsinkwithbackchannel + +- Each profile has it's own codec. However, one problem to solve is, one codec for one direction and a different codec for the other direction. For eg. aptX-LL uses aptX for playback and SBC for recording. How to solve this? Is introducing the backchannel profiles, be sufficient to take care of this? + +### Under the hood + +- https://gavv.github.io/articles/pulseaudio-under-the-hood + +### Bluetooth testing + +``` bash +./simple-endpoint 2C:DB:07:58:F2:F9 ldacsource +./test-device connect 4C:BC:98:80:01:9B "0000110b-0000-1000-8000-00805f9b34fb" +env GST_DEBUG=3 gst-launch-1.0 audiotestsrc ! ldacenc ! a2dpsink transport=/org/bluez/hci0/dev_4C_BC_98_80_01_9B/sep10/fd0 +``` + +## Module loop back + +- https://endless.ersoft.org/pulseaudio-loopback/ diff --git a/purescript.md b/purescript.md new file mode 100644 index 0000000..40d68f0 --- /dev/null +++ b/purescript.md @@ -0,0 +1,5 @@ +## Numeric Heirarchy + +- https://harry.garrood.me/numeric-hierarchy-overview/ + +- https://a-guide-to-the-purescript-numeric-hierarchy.readthedocs.io/en/latest/index.html diff --git a/rust/rust.md b/rust/rust.md new file mode 100644 index 0000000..a1646fa --- /dev/null +++ b/rust/rust.md @@ -0,0 +1,228 @@ +--- +title: Rust +--- + +# Programming Rust + +## Chapter 3 + +- Treats characters as distinct from the numeric types; a char is neither a u8 nor an i8 + +- Requires array indices to be usize + +- In debug builds, rust checks for integer overflow in arithmetic. In release build, the addition would wrap to a negative number. When we want wrapping arithmetic, use methods + + ``` rust + let x = big_val.wrapping_add(1); + ``` + +- Rust currently permits an extra trailing comma everywhere commas are used: function arguments, arrays, struct and enum definitions + +### Arrays, Vectors and Slices + +- Given a value v of any of `Arrays`, `Vectors` or `Slices`, `i` in `v[i]` must be a `usize` value, can't use any other integer type as index + +- Rust has no notion of an uninitialized array + +- The type `[T; N]` represents an array of N values, each of type T. An array's size is a constant determined at compile time, and is part of the type; you can't append new elements or shrink an array. + +- The type `Vec`, called a vector of `Ts`, is a dynamically allocated, growable sequence of values of type T. A vector's elements live on the heap, so you can resize vectors at will. + +- The types `&[T]` and `&mut[T]`, called a shared slice of `Ts` and mutable slice of `Ts`, are references to a series of elements that are part of some other value, like an array or vector. One can think of a slice as a pointer to it's first element, together with the count of the number of elements one can access starting at that point. A mutable slice `&mut [T]` lets one read and modify elements, but can't be shared; a shared slice `&[T]` lets one share access among several readers, but doesn't let one modify elements. + +- The useful methods one did likes to see on arrays - iterating over elements, searching, sorting, filling, filtering and so on - all appear as methods of slices, not arrays. But, Rust implicitly converts a reference to an array to a slice when searching for methods, so you can call any slice method on an array directly. + +1. Slices + + - A slice, written `[T]` without specifying the length, is a region of an array or vector. Since a slice can be any length, slices cannot be stored directly in variables or passed as function arguments. + + - A reference to a slice is a *fat pointer*: a two word value comprising a pointer to the slice's first element, and the number of elements in the slice. + + - Whereas an ordinary reference is a non-owning pointer to a single value, a reference to a slice is non-owning pointer to several values. This makes slices a good choice when you want to write a function that operates on any homogenous data series, whether stored in an array, vector, stack or heap. + + - For example, here's a function that prints a slice of numbers, one per line: + + ``` rust + fn print (n: &[f64]) { + for elt in n { + println!("{}", elt); + } + } + ``` + + - Because the above function takes a slice reference as an argument, you can apply it to either a vector or an array. + +### String + +- For creating new string at run time use `String`. +- The type `&mut str` does exist, but it is not useful, since almost any operation on UTF-8 can change its overall byte length, and a slice cannot reallocate its referent. + +## Chapter 4 - Ownership + +- As a rule of thumb, any type that needs to do something special when a value is dropped cannot be copy. +- By default, `struct` and `enum` types are not `Copy`. + +### Reference as Values + +- In C++, references are created implicitly by conversion, and dereferenced implicitly too. + + let x = 10; + int &r = x; + assert (r == 10); + r = 20; + +- In Rust, references are created explicity with the `&` operator, and deferenced explicity with the `*` operator. + + ``` rust + let x = 10; + let r = &x; + assert!(*r == 10); + ``` + +- Since references are widely used in Rust, the `.` operator implicitly deferences its left operand. + +- The `.` operator can also implicity borrow a reference to its left operand, if needed for a method call. + +- Where as C++ converts implicitly between references and lvalues (that is, expressions referring to locations in memory), with these conversions appearing anywhere they are needed, in Rust you use the `&` and `*` operators to create and follow references, with the exception of `.` operator, which borrows and dereferences implicitly. + +- In C++, assigning to a reference stores the value in its referent. There is no way to point a C++ reference to a location other than the one it was initialized with. + +### References to References + +- Rust permits references to references. + +``` rust +struct Point { x: i32, y: i32 } +let point = Point { x: 1000, y: 729 }; +let r: &Point = &point; +let rr: &&Point = &r; +let rrr: &&&Point = &rr; +``` + +- The `.` operator follows as many references as it takes to find its target. So an expression `rrr.y`, guided by the type of `rrr`, actually traverses three references to get to the `Point` before fetching its `y` field. + +### Sharing vs Mutation + +- Shared access is read only access. +- Mutable access is exclusive access. + +## Chapter 6 - Expressions + +### Functions and Method calls + +One quirk of Rust syntax is that in a function call or method call, the usual syntax for generic types, `Vec`, does not work. + +``` rust +return Vec::with_capacity(1000); // Error: Something about chained comparisons + +let ramp = (0 .. n).collect>(); // Same error +``` + +The problems is that in expressions, `<` is the less-than operator. The Rust compiler helpfully suggests writing `::` in this case, and that solves the problem: + +``` rust +return Vec::::with_capacity(1000); + +let ramp = (0 .. n).collect::>(); +``` + +The symbol `::<...>` is affectionately known in the Rust community as the *turbofish*. + +Alternatively, it is often possible to drop the type parameters and let Rust infer them. + +``` rust +return Vec::with_capacity(1000); + +let ramp : Vec = (0 .. n).collect(); +``` + +It's considered good style to omit the types whenever they can be inferred. + +### Fields and elements + +Rust ranges are half open: they include the start value, if any, but not the end value. + +### Reference operators + +The unary `*` operator is used to access the value pointed to by a reference. Rust automatically follows references when one uses the `.` operator to access a field or method, so the `*` operator is necessary only when we want to read or write the entire value that the reference points to. + +For example, sometimes an iterator produces references, but the program +needs the underlying values: + +``` rust +let padovan: Vec = compute_padovan_sequence(n); + +for elem in &padovan { + draw_triangle(turtle, *elem); +} +``` + +In this example, the type of `elem` is `&u64`, so `*elem` is a `u64`. + +## Chapter 7 - Error Handling + +### Result Type Aliases + +Sometimes you will see Rust documentation that seems to omit the error type of a `Result`: + +``` rust +fn remove_fule(path: &Path) -> Result<()> +``` + +This means a `Result` type alias is being used. + +A type alias is a kind of shorthand for type names. Modules often define a `Result` type alias to avoid having to repeat an error type that's used consistently by almost every function in the module. For example, the standard library's `std::io` module includes this line of code. + +``` rust +pub type Result = result::Result; +``` + +This defines a public type `std::io::Result`. It's an alias for `Result`, but hardcoding `std::io::Error` as the error type. In practical terms, this means that if you write `use std::io`, then Rust will understand `io::Result` as shorthand for `Result`. + +When something like `Result<>` appears in the online documentation, you can click on the identifier `Result` to see which type alias is being used and learn the error type. In practice, it's usually obvious from the context. + +### Chapter 8 - Crates and Modules + +### Chapter 9 - Structs + +The fact that any type can have methods is one reason Rust doesn't use the term `object` much, preferring to call everything a `value`. + +### Chapter 10 - Enums and Patterns + +### Chapter 11 - Traits and Generics + +### Chapter 13 - Utility Traits + +1. Drop + + - If a type implements `Drop`, it cannot implement the `Copy` trait. + +2. Deref and DerefMut + + - Rust doesn't try deref coercions to satisfy type variable bounds. + +### Chapter 14 - Closures + +- Rust can infer the argument type and return type from how the closure is used. + +- Closures do not have the same type as functions. + +- Every closure has its own type, because a closure may contain data: values either borrowed or stolen from enclosing scopes. This can be any number of variables, in any combination of types. So every closure has an ad hoc type created by the compiler, large enough to hold that data. No two closures have exactly the same type. But every closure implements a `Fn` trait. + +- Any closure that requires `mut` access to a vvalue, but doesn't drop any values, is a `FnMut` closure. + +- Closures summary + + - `Fn` is the family of closures and functions that one can call multiple times without restriction. This highest category also includes all functions. + + - `FnMut` is the family of closures that can be called multiple times if the closure itself is declared `mut`. + + - `FnOnce` is the family of closures that can be called once, if the caller owns the closure. + +# Articles on async-await + +- https://os.phil-opp.com/async-await/ +- https://msarmi9.github.io/posts/async-rust/ +- https://fasterthanli.me/articles/pin-and-suffering +- https://smb374.github.io/an-adventure-in-rust-s-async-runtime-model.html +- https://notes.eatonphil.com/lua-in-rust.html diff --git a/toread.md b/toread.md new file mode 100644 index 0000000..509f6f9 --- /dev/null +++ b/toread.md @@ -0,0 +1,31 @@ +# Books + +- [ ] Plato - Last day of Socrates +- [ ] Aristotle - Nichomachean Ethics +- [ ] Epictetus - Discourses, Fragments, Handbook +- [ ] Augustine of Hippo - Confessions +- [ ] Boethius - Consolation of Philosophy +- [ ] Anselm of Canterbury - Three Philosophical Dialogues +- [ ] Thomas Aquinas - Selected Writings +- [ ] Rene Descartes - Meditations on First Philosophy +- [ ] Mary Wollstonecraft - Vindication of the Rights of Women +- [ ] Friedrich Nietzsche - The Genealogy of Morals +- [ ] At the Existentialist Cafe - Sarah Bakewell +- [ ] Republic - Plato + +# News + +- [ ] https://www.theguardian.com/world/2021/jan/14/carbon-neutrality-is-a-fairy-tale-how-the-race-for-renewables-is-burning-europes-forests + +# Articles + +- [ ] https://dbp-consulting.com/tutorials/debugging/linuxProgramStartup.html +- [ ] https://googleprojectzero.blogspot.com/2021/10/how-simple-linux-kernel-memory.html +- [ ] https://blog.reverberate.org/2021/04/21/musttail-efficient-interpreters.html + +# Compilers + +- [ ] [Writing a C Compiler](https://norasandler.com/archive/) +- [ ] [Writing your own VM](https://justinmeiners.github.io/lc3-vm/) +- [ ] [Compiling to assembly from scatch](https://keleshev.com/compiling-to-assembly-from-scratch/) +- [ ] [Crafting Interpreters](https://craftinginterpreters.com/)