[Research] Is Gram's syntax truly language-agnostic? (RTL, CJK, Word Order) #4

New issue

Open

opened 2026-07-15 09:39:54 +02:00 by abiwab · 0 comments

abiwab commented

2026-07-15 09:39:54 +02:00

(Migrated from codeberg.org)

Context

Because Gram is a markup language designed to be woven directly into natural language sentences, it inherently interacts with the grammar and syntax of the language being written.

Currently, Gram's syntax (@, ~, ^, {}, ->) has been primarily tested with English and French. These languages share common traits: Left-to-Right (LTR) reading, spaces between words, and generally a Subject-Verb-Object (SVO) or imperative sentence structure.

The Goal

As I expand Gram's internationalization (i18n), I need to ensure that the parser doesn't break—and that the writing experience doesn't feel unnatural—when used with languages that follow different linguistic rules.

Areas of Investigation

I am looking for feedback and testing from the community on the following potential friction points:

1. Languages without spaces (e.g., Chinese, Japanese)

In languages where words are not separated by spaces, does the parser correctly identify tokens?
Example: 加入@面粉{200 g}和@水{100 ml}. Will the parser fail because there is no space before the @ or after the }?

2. Right-To-Left (RTL) languages (e.g., Arabic, Hebrew)

When mixing RTL text with Left-to-Right (LTR) markup symbols like @, {}, or ->, the bidirectional (BiDi) algorithm of text editors can sometimes break.
Do the curly braces appear inverted in the VS Code extension or the Playground? Does the text cursor behave unpredictably?

3. Word Order & The `[Action]` tag

English recipes usually start with the verb: [Mix] the @flour and @water.
In Subject-Object-Verb (SOV) languages (like Japanese or Turkish), the verb comes at the end.
Does placing the [Action] tag at the beginning of a line feel unnatural? Does the Gram parser force you to put it there, or is it fluid enough to be placed at the end?

4. Complex Pluralization

Currently, Gram's unit dictionaries handle singular/plural pairs (e.g., cup / cups). Some languages (like Russian or Arabic) have complex pluralization rules depending on the exact number (dual, paucal, etc.).
Will the current i18n implementation scale to these rules when rendering recipes?

How to help

If you speak a non-European language or a language with different syntactical rules:

Try writing a simple recipe in the Gram playground or in VS Code using your native language.
Report back here: Did the parser throw errors? Did the text render weirdly? Did the syntax feel ergonomically wrong?

Note: This is a long-term background topic. The goal right now is simply to gather data to determine if supporting these linguistic edge cases is genuinely necessary for everyday use, and if it's technically feasible without overcomplicating the core parser.

I'd love to hear your thoughts on this topic and eventually see your test cases.

## Context Because Gram is a markup language designed to be woven directly into natural language sentences, it inherently interacts with the grammar and syntax of the language being written. Currently, Gram's syntax (`@`, `~`, `^`, `{}`, `->`) has been primarily tested with English and French. These languages share common traits: Left-to-Right (LTR) reading, spaces between words, and generally a Subject-Verb-Object (SVO) or imperative sentence structure. ## The Goal As I expand Gram's internationalization (i18n), I need to ensure that the parser doesn't break—and that the writing experience doesn't feel unnatural—when used with languages that follow different linguistic rules. ## Areas of Investigation I am looking for feedback and testing from the community on the following potential friction points: ### 1. Languages without spaces (e.g., Chinese, Japanese) In languages where words are not separated by spaces, does the parser correctly identify tokens? *Example: `加入@面粉{200 g}和@水{100 ml}`. Will the parser fail because there is no space before the `@` or after the `}`?* ### 2. Right-To-Left (RTL) languages (e.g., Arabic, Hebrew) When mixing RTL text with Left-to-Right (LTR) markup symbols like `@`, `{}`, or `->`, the bidirectional (BiDi) algorithm of text editors can sometimes break. *Do the curly braces appear inverted in the VS Code extension or the Playground? Does the text cursor behave unpredictably?* ### 3. Word Order & The `[Action]` tag English recipes usually start with the verb: `[Mix] the @flour and @water`. In Subject-Object-Verb (SOV) languages (like Japanese or Turkish), the verb comes at the end. *Does placing the `[Action]` tag at the beginning of a line feel unnatural? Does the Gram parser force you to put it there, or is it fluid enough to be placed at the end?* ### 4. Complex Pluralization Currently, Gram's unit dictionaries handle singular/plural pairs (e.g., `cup` / `cups`). Some languages (like Russian or Arabic) have complex pluralization rules depending on the exact number (dual, paucal, etc.). *Will the current i18n implementation scale to these rules when rendering recipes?* ## How to help If you speak a non-European language or a language with different syntactical rules: 1. **Try writing a simple recipe** in the Gram playground or in VS Code using your native language. 2. **Report back here:** Did the parser throw errors? Did the text render weirdly? Did the syntax feel ergonomically wrong? *Note: This is a long-term background topic. The goal right now is simply to gather data to determine if supporting these linguistic edge cases is genuinely necessary for everyday use, and if it's technically feasible without overcomplicating the core parser.* I'd love to hear your thoughts on this topic and eventually see your test cases.