node: Add config file support (#3710)

* node: Add logic to read file config and bind flags

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: add guardian node config to node.yaml

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: fix path typo

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: modularize initFileConfig

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: update ethRPC to the correct url

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: update config file path

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: add initial config file testing data

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: add test for flag precedence over config file

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: add test cases for flag, env var and config file precedence

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: use backticks as expected output

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: update comments

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: handle binding errors

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: create separate test functions

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: absolute filepath -> relative filepath

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* docs: Add guardian config file usage

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

* node: update config file name and env var prefix

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

---------

Signed-off-by: bingyuyap <bingyu.yap.21@gmail.com>

devnet/node.yaml

@@ -60,6 +60,9 @@ spec:
                 path: gwrelayerKey0
               - key: gwrelayerKey1
                 path: gwrelayerKey1
+        - name: node-config
+          configMap:
+            name: node-config
       containers:
         - name: guardiand
           image: guardiand-image
@@ -68,11 +71,13 @@ spec:
               name: node-rundir
             - mountPath: /tmp/mounted-keys/wormchain
               name: node-wormchain-key
+            - mountPath: /app/node/config
+              name: node-config
           command:
             - /guardiand
             - node
-            - --ethRPC
-            - ws://eth-devnet:8545
+            # - --ethRPC
+            # - ws://eth-devnet:8545
             # - --bscRPC
             # - ws://eth-devnet2:8545
             - --polygonRPC
@@ -219,3 +224,11 @@ data:
   accountantKey1: LS0tLS1CRUdJTiBURU5ERVJNSU5UIFBSSVZBVEUgS0VZLS0tLS0Ka2RmOiBiY3J5cHQKc2FsdDogNzc1M0NCQTBBMUQ0NTJCMkE2QzlERDM4ODc3MTg0NEEKdHlwZTogc2VjcDI1NmsxCgpSYnhRVWRnK2ZHcjMzZTAyVWFFQW1YTDFlNFkrTGJUMFdqbnl4RVR3OXBoL2JXOGI0MzdhWmErOWlCc3NBa0UyCnRScUwvb0J1NWFnQXJocHNnWUgxNlhOWjJHMXRwY0R3V0dQZ1VWVT0KPUd6YUwKLS0tLS1FTkQgVEVOREVSTUlOVCBQUklWQVRFIEtFWS0tLS0t
   gwrelayerKey0: LS0tLS1CRUdJTiBURU5ERVJNSU5UIFBSSVZBVEUgS0VZLS0tLS0KdHlwZTogc2VjcDI1NmsxCmtkZjogYmNyeXB0CnNhbHQ6IDc4OUYzRTBCMkVGNDcyNjAyQzNFMUE0OUI2OENFQzlBCgpGWHAvSllPS3E4WmZtOWxHZ3ZFNEM3NXFyUXFNZFp2RHNWRjhObTdMQU1oR2dHbXBnZnpoZjUrZ3IwZ1hjYjVWCmtSTXA2c0p0NkxCVzRPYWF2ckk3ay84Vml2NWhMVU1la1dPMHg5bz0KPUxrb1MKLS0tLS1FTkQgVEVOREVSTUlOVCBQUklWQVRFIEtFWS0tLS0t
   gwrelayerKey1: LS0tLS1CRUdJTiBURU5ERVJNSU5UIFBSSVZBVEUgS0VZLS0tLS0Ka2RmOiBiY3J5cHQKc2FsdDogNDc5RDk3RDE2OTE0QkQ4QjlFNUUwQzkzMDA0RDA4RUEKdHlwZTogc2VjcDI1NmsxCgpvTEJ0aUkwT2pudXo5bHlzeVlZOFhQeEVkTnpwYUJOVWFkL0UySlJld2pFWFZNVVNTWll2QVZKbERiN3hEQjlSCmEvdm45SFNPM2hKOFc1QTBKOVFqUVZXRzVoZXBNZVpQUEI4M1FCUT0KPVJuTGEKLS0tLS1FTkQgVEVOREVSTUlOVCBQUklWQVRFIEtFWS0tLS0t
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: node-config
+data:
+  guardiand.yaml: |
+    ethRPC: "ws://eth-devnet:8545"

docs/operations.md

@@ -300,3 +300,48 @@ docker run \
     ghcr.io/wormhole-foundation/guardiand:latest \
 spy --nodeKey /node.key --spyRPC "[::]:7073" --network /wormhole/mainnet/2 --bootstrap /dns4/wormhole-v2-mainnet-bootstrap.xlabs.xyz/udp/8999/quic/p2p/12D3KooWNQ9tVrcb64tw6bNs2CaNrUGPM7yRrKvBBheQ5yCyPHKC,/dns4/wormhole.mcf.rocks/udp/8999/quic/p2p/12D3KooWDZVv7BhZ8yFLkarNdaSWaB43D6UbQwExJ8nnGAEmfHcU,/dns4/wormhole-v2-mainnet-bootstrap.staking.fund/udp/8999/quic/p2p/12D3KooWG8obDX9DNi1KUwZNu9xkGwfKqTp2GFwuuHpWZ3nQruS1
 ```
+
+## Guardian Configurations
+
+Configuration files, environment variables and flags are all supported.
+
+### Config File
+
+**Location/Naming**: By default, the config file is expected to be in the `node/config` directory. The standard name for the config file is `guardiand.yaml`. Currently there's no support for custom directory or filename yet.
+
+**Format**: We support any format that is supported by [Viper](https://pkg.go.dev/github.com/dvln/viper#section-readme). But YAML format is generally preferred.
+
+**Example**:
+```yaml
+ethRPC: "ws://eth-devnet:8545"
+ethContract: "0xC89Ce4735882C9F0f0FE26686c53074E09B0D550"
+solanaRPC: "http://solana-devnet:8899"
+solanaContract: "Bridge1p5gheXUvJ6jGWGeCsgPKgnE3YgdGKRVCMY9o"
+```
+
+### Environment Variables
+
+**Prefix**: All environment variables related to the Guardian node should be prefixed with `GUARDIAND_`.
+
+**Usage**: Environment variables can be used to override settings in the config file. Particularly for sensitive data like API keys that should not be stored in config files.
+
+**Example**:
+```bash
+export GUARDIAND_ETHRPC=ws://eth-devnet:8545
+```
+
+### Command-Line Flags
+
+**Usage**: Flags provide the highest precedence and can be used for temporary overrides or for settings that change frequently.
+
+**Example**:
+```bash
+./guardiand node --ethRPC=ws://eth-devnet:8545
+```
+
+### Precedence Order
+The configuration settings are applied in the following order of precedence:
+
+1. **Command-Line Flags**: Highest precedence, overrides any other settings.
+2. **Environment Variables**: Overrides the config file settings but can be overridden by flags.
+3. **Config File**: Lowest precedence.

node/cmd/guardiand/node.go

@@ -417,6 +417,12 @@ var (
 	rootCtxCancel context.CancelFunc
 )
 
+var (
+	configFilename = "guardiand"
+	configPath     = "node/config"
+	envPrefix      = "GUARDIAND"
+)
+
 // "Why would anyone do this?" are famous last words.
 //
 // We already forcibly override RPC URLs and keys in dev mode to prevent security
@@ -432,9 +438,10 @@ const devwarning = `
 
 // NodeCmd represents the node command
 var NodeCmd = &cobra.Command{
-	Use:   "node",
-	Short: "Run the guardiand node",
-	Run:   runNode,
+	Use:               "node",
+	Short:             "Run the guardiand node",
+	PersistentPreRunE: initConfig,
+	Run:               runNode,
 }
 
 // This variable may be overridden by the -X linker flag to "dev" in which case
@@ -443,6 +450,15 @@ var NodeCmd = &cobra.Command{
 // guardians to reduce risk from a compromised builder.
 var Build = "prod"
 
+// initConfig initializes the file configuration.
+func initConfig(cmd *cobra.Command, args []string) error {
+	return node.InitFileConfig(cmd, node.ConfigOptions{
+		FilePath:  configPath,
+		FileName:  configFilename,
+		EnvPrefix: envPrefix,
+	})
+}
+
 func runNode(cmd *cobra.Command, args []string) {
 	if Build == "dev" && !*unsafeDevMode {
 		fmt.Println("This is a development build. --unsafeDevMode must be enabled.")

node/pkg/node/config_file_reader.go

@@ -0,0 +1,62 @@
+package node
+
+import (
+	"fmt"
+	"log"
+
+	"github.com/spf13/cobra"
+	"github.com/spf13/pflag"
+	"github.com/spf13/viper"
+)
+
+type ConfigOptions struct {
+	FilePath  string
+	FileName  string
+	EnvPrefix string
+}
+
+// InitFileConfig initializes configuration according to the following precedence:
+// 1. Command line flags
+// 2. Environment variables
+// 3. Config file
+// 4. Cobra default values
+func InitFileConfig(cmd *cobra.Command, options ConfigOptions) error {
+	v := viper.New()
+
+	v.SetConfigName(options.FileName)
+	v.AddConfigPath(options.FilePath)
+
+	if err := v.ReadInConfig(); err != nil {
+		if _, ok := err.(viper.ConfigFileNotFoundError); !ok {
+			return err
+		}
+	}
+
+	// Bind flags to environment variables with a common prefix to avoid conflicts
+	// Example: --ethRPC will be bound to GUARDIAND_ETHRPC
+	v.SetEnvPrefix(options.EnvPrefix)
+
+	// Bind to environment variables
+	v.AutomaticEnv()
+
+	// Bind the current command's flags to viper
+	bindFlags(cmd, v)
+
+	return nil
+}
+
+func bindFlags(cmd *cobra.Command, v *viper.Viper) {
+	cmd.Flags().VisitAll(func(f *pflag.Flag) {
+		// Determine the naming convention of the flags when represented in the config file
+		configName := f.Name
+
+		// Apply the viper config value to the flag when the flag is not set and viper has a value
+		if !f.Changed && v.IsSet(configName) {
+			val := v.Get(configName)
+			err := cmd.Flags().Set(f.Name, fmt.Sprintf("%v", val))
+			if err != nil {
+				log.Fatalf("failed to bind flag %s to viper: %v", f.Name, err)
+			}
+		}
+	})
+}

node/pkg/node/config_file_reader_test.go

@@ -0,0 +1,105 @@
+package node
+
+import (
+	"bytes"
+	"fmt"
+	"os"
+	"testing"
+
+	"github.com/spf13/cobra"
+	"github.com/stretchr/testify/assert"
+)
+
+func NewTestRootCommand() *cobra.Command {
+	var ethRPC *string
+	var solRPC *string
+
+	// Define test configuration
+	testConfig := ConfigOptions{
+		FilePath:  "testdata",
+		FileName:  "test",
+		EnvPrefix: "TEST_GUARDIAND",
+	}
+
+	rootCmd := &cobra.Command{
+		Use:   "config_file_reader_test",
+		Short: "Unit test to test config file reader",
+		PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
+			// Initialize configuration using Viper
+			return InitFileConfig(cmd, testConfig) // Adjust the filename as needed
+		},
+		Run: func(cmd *cobra.Command, args []string) {
+			// Working with OutOrStdout/OutOrStderr allows us to unit test our command easier
+			out := cmd.OutOrStdout()
+
+			// Print the final resolved value from binding cobra flags and viper config
+			fmt.Fprintln(out, "ethRPC:", *ethRPC)
+			fmt.Fprintln(out, "solRPC:", *solRPC)
+		},
+	}
+
+	ethRPC = rootCmd.Flags().String("ethRPC", "", "Ethereum RPC URL")
+	solRPC = rootCmd.Flags().String("solRPC", "", "Solana RPC URL")
+
+	return rootCmd
+}
+
+// Set ethRPC with config file
+// Tests that the config file is read and the default value is set
+func TestInitFileConfig(t *testing.T) {
+
+	cmd := NewTestRootCommand()
+	output := &bytes.Buffer{}
+	cmd.SetOut(output)
+	_ = cmd.Execute()
+
+	gotOutput := output.String()
+	wantOutput := `ethRPC: ws://eth-config-file:8545
+solRPC: ws://sol-config-file:8545
+`
+	assert.Equal(t, wantOutput, gotOutput, "expected ethRPC to use the config file default")
+}
+
+// Set ethRPC with an environment variable
+// Tests that environment variables take precedence over config file values
+func TestEnvVarPrecedence(t *testing.T) {
+	os.Setenv("TEST_GUARDIAND_ETHRPC", "ws://eth-env-var:8545")
+	defer os.Unsetenv("TEST_GUARDIAND_ETHRPC")
+
+	cmd := NewTestRootCommand()
+	output := &bytes.Buffer{}
+	cmd.SetOut(output)
+	_ = cmd.Execute()
+
+	gotOutput := output.String()
+	wantOutput := `ethRPC: ws://eth-env-var:8545
+solRPC: ws://sol-config-file:8545
+`
+
+	assert.Equal(t, wantOutput, gotOutput, "expected ethRPC to use the environment variable and solRPC to use the config file default")
+}
+
+// Set ethRPC with a flag
+// Tests that flags take precedence over environment variables and config file values
+func TestFlagPrecedence(t *testing.T) {
+	os.Setenv("TEST_GUARDIAND_ETHRPC", "ws://eth-env-var:8545")
+	defer os.Unsetenv("TEST_GUARDIAND_ETHRPC")
+
+	cmd := NewTestRootCommand()
+	output := &bytes.Buffer{}
+	cmd.SetOut(output)
+	cmd.SetArgs([]string{
+		"--ethRPC",
+		"ws://eth-flag:8545",
+		"--solRPC",
+		"ws://sol-flag:8545",
+	})
+	_ = cmd.Execute()
+
+	gotOutput := output.String()
+	wantOutput := `ethRPC: ws://eth-flag:8545
+solRPC: ws://sol-flag:8545
+`
+
+	assert.Equal(t, wantOutput, gotOutput, "expected the ethRPC to use the flag value and solRPC to use the flag value")
+}

node/pkg/node/testdata/test.yaml

@@ -0,0 +1,2 @@
+ethRPC: "ws://eth-config-file:8545"
+solRPC: "ws://sol-config-file:8545"