[docs] translation support (#35166)

* chore: update crowdin and add serve command

* feat: upload all desired files to crowdin

* fix: absolute urls for pdfs

* fix: do not import components with relative paths

* feat: updated readme

* fix: whitespace

docs/README.md

@@ -63,8 +63,12 @@ npm run start
 
 ## Translations
 
-Translations are sourced from [Crowdin](https://docusaurus.io/docs/i18n/crowdin) and generated when `master` is built.
-For local development use the following two commands in this `docs` directory.
+Translations are sourced from [Crowdin](https://docusaurus.io/docs/i18n/crowdin)
+and generated when the branch noted as the `STABLE` channel is built via the
+`build.sh` script.
+
+For local development, and with the `CROWDIN_PERSONAL_TOKEN` env variable set,
+use the following two commands in this `docs` directory.
 
 To download the newest documentation translations run:
 
@@ -72,12 +76,45 @@ To download the newest documentation translations run:
 npm run crowdin:download
 ```
 
-To upload changes from `src` & generate [explicit IDs](https://docusaurus.io/docs/markdown-features/headings#explicit-ids):
+To upload changes from `src` & generate
+[explicit IDs](https://docusaurus.io/docs/markdown-features/headings#explicit-ids):
 
 ```shell
 npm run crowdin:upload
 ```
 
+> Translations are only included when deploying the `STABLE` channel of the docs
+> (via `build.sh`). Resulting in only the `docs.solanalabs.com` documentation
+> site to include translated content. Therefore, the `edge` and `beta` docs
+> sites are not expected to include translated content, even though the language
+> selector will still be present.
+
+### Common issues
+
+#### `CROWDIN_PERSONAL_TOKEN` env variable
+
+The `crowdin.yml` file requires a `CROWDIN_PERSONAL_TOKEN` env variable to be
+set with a valid Crowdin access token.
+
+For local development, you can store this in a `.env` file that the Crowdin CLI
+will auto detect.
+
+For building and publishing via the GitHub actions, the `CROWDIN_PERSONAL_TOKEN`
+secret must be set.
+
+#### Translation locale fails to build with `SyntaxError`
+
+Some translation locales may fail to build with a `SyntaxError` thrown by
+Docusaurus due to how certain language symbols get parsed by Docusaurus while
+generating the static version of the docs.
+
+> Note: When any locale fails to build, the entire docs build will fail
+> resulting in the docs not being able to be deployed at all.
+
+There are several known locales that fail to build the current documentation.
+They are listed in the commented out `localesNotBuilding` attribute in the
+[`docusaurus.config.js`](https://github.com/solana-labs/solana/blob/master/docs/docusaurus.config.js)
+
 ## CI Build Flow
 
 The docs are built and published in Travis CI with the `./build.sh` script. On each PR, the docs are built, but not published.

docs/crowdin.yml

@@ -4,13 +4,22 @@ base_url: 'https://solana.crowdin.com'
 preserve_hierarchy: true
 files: [
     # JSON translation files
-    # {
-    # source: '/i18n/en/**/*',
-    # translation: '/i18n/%two_letters_code%/**/%original_file_name%',
-    #},
+    {
+    source: '/i18n/en/**/*',
+    translation: '/i18n/%two_letters_code%/**/%original_file_name%',
+    },
     # Docs Markdown files
     {
       source: '/src/**/*.md',
       translation: '/i18n/%two_letters_code%/docusaurus-plugin-content-docs/current/**/%original_file_name%',
     },
+    {
+      source: '/src/**/*.mdx',
+      translation: '/i18n/%two_letters_code%/docusaurus-plugin-content-docs/current/**/%original_file_name%',
+    },
+    # Custom sidebar category files
+    {
+      source: '/src/**/*.json',
+      translation: '/i18n/%two_letters_code%/docusaurus-plugin-content-docs/current/**/%original_file_name%',
+    },
   ]

docs/package-lock.json

@@ -8,7 +8,7 @@
       "name": "solana-docs",
       "version": "0.0.0",
       "dependencies": {
-        "@crowdin/cli": "^3.6.1",
+        "@crowdin/cli": "^3.17.0",
         "@docusaurus/core": "^2.2.0",
         "@docusaurus/plugin-google-gtag": "^2.4.0",
         "@docusaurus/preset-classic": "^2.2.0",
@@ -2029,12 +2029,15 @@
       }
     },
     "node_modules/@crowdin/cli": {
-      "version": "3.9.0",
-      "resolved": "https://registry.npmjs.org/@crowdin/cli/-/cli-3.9.0.tgz",
-      "integrity": "sha512-4wQjqJZmU/mg3VYfRL6IYXw/pPAL9vdfW3QVSBovYA+bYaEt43ZuGsSrqeBGOhLehasWwRqklXWsl96gxQlLdw==",
+      "version": "3.17.0",
+      "resolved": "https://registry.npmjs.org/@crowdin/cli/-/cli-3.17.0.tgz",
+      "integrity": "sha512-ipr5wyBvpVuJ/DtJgDqTJiECu7zsVn9DwyTdf+sa0ukksXyiX3+H6wPm4eefIfEVSEaM92Q572dJZ5OnIH/Sag==",
       "dependencies": {
-        "njre": "^0.2.0",
-        "shelljs": "^0.8.4"
+        "command-exists-promise": "^2.0.2",
+        "node-fetch": "2.6.7",
+        "shelljs": "^0.8.4",
+        "tar": "^4.4.8",
+        "yauzl": "^2.10.0"
       },
       "bin": {
         "crowdin": "jdeploy-bundle/jdeploy.js"
@@ -9570,20 +9573,6 @@
       "resolved": "https://registry.npmjs.org/neo-async/-/neo-async-2.6.2.tgz",
       "integrity": "sha512-Yd3UES5mWCSqR+qNT93S3UoYUkqAZ9lLg8a7g9rimsWmYGK8cVToA4/sF3RrshdyV3sAGMXVUmpMYOw+dLpOuw=="
     },
-    "node_modules/njre": {
-      "version": "0.2.0",
-      "resolved": "https://registry.npmjs.org/njre/-/njre-0.2.0.tgz",
-      "integrity": "sha512-+Wq8R6VmjK+jI8a9NdzfU6Vh50r3tjsdvl5KJE1OyHeH8I/nx5Ptm12qpO3qNUbstXuZfBDgDL0qQZw9JyjhMw==",
-      "dependencies": {
-        "command-exists-promise": "^2.0.2",
-        "node-fetch": "^2.5.0",
-        "tar": "^4.4.8",
-        "yauzl": "^2.10.0"
-      },
-      "engines": {
-        "node": ">=8"
-      }
-    },
     "node_modules/no-case": {
       "version": "3.0.4",
       "resolved": "https://registry.npmjs.org/no-case/-/no-case-3.0.4.tgz",
@@ -15762,12 +15751,15 @@
       "optional": true
     },
     "@crowdin/cli": {
-      "version": "3.9.0",
-      "resolved": "https://registry.npmjs.org/@crowdin/cli/-/cli-3.9.0.tgz",
-      "integrity": "sha512-4wQjqJZmU/mg3VYfRL6IYXw/pPAL9vdfW3QVSBovYA+bYaEt43ZuGsSrqeBGOhLehasWwRqklXWsl96gxQlLdw==",
+      "version": "3.17.0",
+      "resolved": "https://registry.npmjs.org/@crowdin/cli/-/cli-3.17.0.tgz",
+      "integrity": "sha512-ipr5wyBvpVuJ/DtJgDqTJiECu7zsVn9DwyTdf+sa0ukksXyiX3+H6wPm4eefIfEVSEaM92Q572dJZ5OnIH/Sag==",
       "requires": {
-        "njre": "^0.2.0",
-        "shelljs": "^0.8.4"
+        "command-exists-promise": "^2.0.2",
+        "node-fetch": "2.6.7",
+        "shelljs": "^0.8.4",
+        "tar": "^4.4.8",
+        "yauzl": "^2.10.0"
       }
     },
     "@cspotcode/source-map-support": {
@@ -21261,17 +21253,6 @@
       "resolved": "https://registry.npmjs.org/neo-async/-/neo-async-2.6.2.tgz",
       "integrity": "sha512-Yd3UES5mWCSqR+qNT93S3UoYUkqAZ9lLg8a7g9rimsWmYGK8cVToA4/sF3RrshdyV3sAGMXVUmpMYOw+dLpOuw=="
     },
-    "njre": {
-      "version": "0.2.0",
-      "resolved": "https://registry.npmjs.org/njre/-/njre-0.2.0.tgz",
-      "integrity": "sha512-+Wq8R6VmjK+jI8a9NdzfU6Vh50r3tjsdvl5KJE1OyHeH8I/nx5Ptm12qpO3qNUbstXuZfBDgDL0qQZw9JyjhMw==",
-      "requires": {
-        "command-exists-promise": "^2.0.2",
-        "node-fetch": "^2.5.0",
-        "tar": "^4.4.8",
-        "yauzl": "^2.10.0"
-      }
-    },
     "no-case": {
       "version": "3.0.4",
       "resolved": "https://registry.npmjs.org/no-case/-/no-case-3.0.4.tgz",

docs/package.json

@@ -5,6 +5,7 @@
   "scripts": {
     "start": "docusaurus start",
     "build": "docusaurus build",
+    "serve": "docusaurus serve",
     "clear": "docusaurus clear",
     "help": "docusaurus --help",
     "swizzle": "docusaurus swizzle",
@@ -20,7 +21,7 @@
     "crowdin:upload": "npm run write-i18n && crowdin upload"
   },
   "dependencies": {
-    "@crowdin/cli": "^3.6.1",
+    "@crowdin/cli": "^3.17.0",
     "@docusaurus/core": "^2.2.0",
     "@docusaurus/plugin-google-gtag": "^2.4.0",
     "@docusaurus/preset-classic": "^2.2.0",

docs/src/index.mdx

@@ -55,6 +55,6 @@ Explore what it takes to operate a Solana validator and help secure the network.
 
 ## Learn more
 
-import HomeCtaLinks from "../components/HomeCtaLinks";
+import HomeCtaLinks from "@site/components/HomeCtaLinks";
 
 <HomeCtaLinks />

docs/src/runtime/zk-token-proof.md

@@ -41,7 +41,8 @@ cannot change the original value that is contained in a commitment.
 Interested readers can refer to the following resources for a more in-depth
 treatment of Pedersen commitment and the (twisted) ElGamal encryption schemes.
 
-- [Notes](./zk-docs/twisted_elgamal.pdf) on the twisted ElGamal encryption
+- [Notes](https://github.com/solana-labs/solana/blob/master/docs/src/runtime/zk-docs/twisted_elgamal.pdf)
+  on the twisted ElGamal encryption
 - A technical
   [overview](https://github.com/solana-labs/solana-program-library/blob/master/token/zk-token-protocol-paper/part1.pdf)
   of the SPL Token 2022 confidential extension
@@ -98,14 +99,14 @@ The ZK Token proof program supports the following list of zero-knowledge proofs.
   - The ElGamal public-key validity proof instruction certifies that an ElGamal
     public-key is a properly formed public key.
   - Mathematical description and proof of security:
-    [[Notes]](./zk-docs/pubkey_proof.pdf)
+    [[Notes]](https://github.com/solana-labs/solana/blob/master/docs/src/runtime/zk-docs/pubkey_proof.pdf)
 
 - `VerifyZeroBalance`:
 
   - The zero-balance proof certifies that an ElGamal ciphertext encrypts the
     number zero.
   - Mathematical description and proof of security:
-    [[Notes]](./zk-docs/zero_proof.pdf)
+    [[Notes]](https://github.com/solana-labs/solana/blob/master/docs/src/runtime/zk-docs/zero_proof.pdf)
 
 #### Equality proofs
 
@@ -114,11 +115,11 @@ The ZK Token proof program supports the following list of zero-knowledge proofs.
   - The ciphertext-commitment equality proof certifies that an ElGamal
     ciphertext and a Pedersen commitment encode the same message.
   - Mathematical description and proof of security:
-    [[Notes]](./zk-docs/ciphertext_commitment_equality.pdf)
+    [[Notes]](https://github.com/solana-labs/solana/blob/master/docs/src/runtime/zk-docs/ciphertext_commitment_equality.pdf)
 
 - `VerifyCiphertextCiphertextEquality`:
 
   - The ciphertext-ciphertext equality proof certifies that two ElGamal
     ciphertexts encrypt the same message.
   - Mathematical description and proof of security:
-    [[Notes]](./zk-docs/ciphertext_ciphertext_equality.pdf)
+    [[Notes]](https://github.com/solana-labs/solana/blob/master/docs/src/runtime/zk-docs/ciphertext_ciphertext_equality.pdf)