5 Commits

Author SHA1 Message Date
Zoe
f7ac354bcf fix config typos and misleading errors 2025-12-06 23:27:54 -06:00
Zoe
dcd8d810f0 More robust script handling and code cleanup
In this commit, I have made the script handling dramatically more
robust, and the code has been cleaned up a fair amount. We no longer
have to manually set is restarting or anything like that, and we dont
rely on a fragile exit code system.
2025-10-07 16:15:05 +00:00
Zoe
4a70260ea5 Better debugging, and several bug fixes
This commit brings a greater experience for developers working on ZQDGR,
and also fixes a number of bugs. The following bugs have been fixed:
- Scripts that died in watch mode did not restart once changes were made
- Created and deleted files did not cause a reload
- script.Stop is now used in every place where we kill the process, this
  ensures that the process is *eventually* actually killed
2025-10-06 15:54:31 -05:00
Zoe
a62bfdb01d Fix restart after program has died 2025-10-06 14:43:33 +00:00
Zoe
f52a61b9ea ZQDGR v0.0.4: Qualirt of Life Update
This update brings several quality of life enhancements:
- You can now use excluded_files to exclude files from the watcher
- ZQDGR reloads itself when its config changes
- ZQDGR will not start if there are unrecognized keys in the config

It also fixes a bug where if a program was sent a kill signal, but didnt
die, ZQDGR would carry on as if it did, which lead to many issues.
2025-10-03 02:00:22 -05:00
5 changed files with 367 additions and 158 deletions

View File

@@ -5,6 +5,13 @@ how you would use npm. ZQDGR lets you watch files and rebuild your project as yo
optional websocket server that will notify listeners that a rebuild has occurred, this is very useful for live reloading
when doing web development with Go.
**Status: Alpha**
This project is not recommend for production use, you are allowed to, and I try
my best to keep it as stable as possible, but features and APIs will change
in nearly every release without backwards compatibility, upgrade carefilly, and
make sure to depend on a specific version of ZQDGR.
## Install
```bash
@@ -14,44 +21,64 @@ go install github.com/juls0730/zqdgr@latest
## Usage
Full usage
```Bash
zqdgr [options] <command>
```
The list of commands is
### Commands
- `init`
generates a zqdgr.config.json file in the current directory
- `new [project name] [github repo]`
Creates a new golang project with zqdgr and can optionally run scripts from a github repo
- `watch <script>`
runs the script in "watch mode", when files that follow the pattern in zqdgr.config.json change, the script restarts
- `<script>`
runs the script
### Options
ZQDGR has the following list of options
- `-no-ws`
disables the web socket server running at 2067
- `-config`
specifies the path to the config file
- `-disable-reload-config`
disables reloading the config file when it changes
- `-C`
specifies the path to use as the working directry
Example usage:
```bash
zqdgr init
zqdgr watch dev
zqdgr -config ./my-config.json watch dev
zqdgr -C ./my-nested-project watch dev
```
### ZQDGR websocket
ZQDGR comes with a websocket to notify listeners that the application has updates, the websocket is accessible at
`127.0.0.1:2067/ws`. An example dev script to listen for rebuilds might look like this
```Javascript
let host = window.location.hostname;
const socket = new WebSocket('ws://' + host + ':2067/ws');
const socket = new WebSocket('ws://' + host + ':2067/ws');
socket.addEventListener('message', (event) => {
if (event.data === 'refresh') {
@@ -75,21 +102,22 @@ socket.addEventListener('message', (event) => {
ZQDGR is solely configured by a `zqdgr.config.json` file in the root of your project. The file has the following keys:
| Key | Type | Description |
| --- | --- | --- |
| name | string | The name of the project |
| version | string | The version of the project |
| description | string | The description of the project |
| author | string | The author of the project (probably you) |
| license | string | The license of the project |
| homepage | string | The URL to the homepage of the project |
| repository | object | The repository of the project |
| repository.type | string | The type of VCS that you use, most likely git |
| repository.url | string | The place where you code is hosted. This should be a URI that can be used as is in a program such as git |
| scripts | object | An object that maps a script name to a script command, which is just a shell command |
| pattern | string | The GLOB pattern that ZQDGR will watch for changes |
| excluded_dirs | array | The directories that ZQDGR will ignore when in the `watch` mode |
| shutdown_signal | string | The signal that ZQDGR will use to shutdown the script. Currently the only supported signals are `SIGINT`, `SIGTERM`, and `SIGQUIT`, if no shutdown signal is specified, ZQDGR will default to `SIGKILL` |
| Key | Type | Required | default | Description |
| ---------------- | ------ | -------- | ------- | -------------------------------------------------------------------------------------------------------- |
| name | string | - | - | The name of the project |
| version | string | - | - | The version of the project |
| description | string | - | - | The description of the project |
| author | string | - | - | The author of the project (probably you) |
| license | string | - | - | The license of the project |
| homepage | string | - | - | The URL to the homepage of the project |
| repository | object | - | - | The repository of the project |
| repository.type | string | - | - | The type of VCS that you use, most likely git |
| repository.url | string | - | - | The place where you code is hosted. This should be a URI that can be used as is in a program such as git |
| scripts | object | true | - | An object that maps a script name to a script command, which is just a shell command |
| pattern | string | true | - | The GLOB pattern that ZQDGR will watch for changes |
| excluded_globs | array | - | - | Globs that ZQDGR will ignore when in the `watch` mode |
| shutdown_signal | string | - | SIGINT | The signal that ZQDGR will use to shutdown the script. One of SIGKILL, SIGINT, SIGQUIT, SIGTERM |
| shutdown_timeout | int | - | 1 | The amount of time in seconds that ZQDGR will wait for the script to exit before force killing it. |
The only required key is `scripts`, the rest are optional, but we recommend you set the most important ones.

View File

@@ -9,5 +9,5 @@
"dev": "go run main.go"
},
"pattern": "**/*.go",
"excluded_dirs": []
"excluded_globs": []
}

383
main.go
View File

@@ -2,11 +2,13 @@ package main
import (
"bufio"
"bytes"
_ "embed"
"encoding/json"
"flag"
"fmt"
"log"
"log/slog"
"net/http"
"os"
"os/exec"
@@ -14,6 +16,7 @@ import (
"path"
"path/filepath"
"runtime"
"strconv"
"strings"
"sync"
"syscall"
@@ -37,21 +40,20 @@ type Config struct {
Type string `json:"type"`
URL string `json:"url"`
} `json:"repository"`
Scripts map[string]string `json:"scripts"`
Pattern string `json:"pattern"`
ExcludedDirs []string `json:"excluded_dirs"`
ShutdownSignal string `json:"shutdown_signal"`
Scripts map[string]string `json:"scripts"`
Pattern string `json:"pattern"`
ExcludedGlobs []string `json:"excluded_globs"`
ShutdownSignal string `json:"shutdown_signal"`
ShutdownTimeout int `json:"shutdown_timeout"`
}
type Script struct {
zqdgr *ZQDGR
command *exec.Cmd
mutex sync.Mutex
scriptName string
isRestarting bool
wg sync.WaitGroup
// the exit code of the script, only set after the script has exited
exitCode int
zqdgr *ZQDGR
command *exec.Cmd
mutex sync.Mutex
scriptName string
// notified with the exit code of the script when it exits
exitCode chan int
}
type ZQDGR struct {
@@ -72,7 +74,11 @@ func NewZQDGR(enableWebSocket bool, configDir string) *ZQDGR {
WorkingDirectory: configDir,
}
zqdgr.loadConfig()
err := zqdgr.loadConfig()
if err != nil {
log.Fatal(err)
return nil
}
zqdgr.EnableWebSocket = enableWebSocket
zqdgr.WSServer = &WSServer{
@@ -124,10 +130,10 @@ func (zqdgr *ZQDGR) NewScript(scriptName string, args ...string) *Script {
}
return &Script{
zqdgr: zqdgr,
command: command,
scriptName: scriptName,
isRestarting: false,
zqdgr: zqdgr,
command: command,
scriptName: scriptName,
exitCode: make(chan int),
}
}
@@ -135,55 +141,94 @@ func (s *Script) Start() error {
s.mutex.Lock()
defer s.mutex.Unlock()
s.wg.Add(1)
err := s.command.Start()
if err != nil {
s.wg.Done()
return err
}
go func() {
err := s.command.Wait()
if err != nil {
if exitError, ok := err.(*exec.ExitError); ok {
s.exitCode = exitError.ExitCode()
} else {
// Other errors (e.g., process not found, permission denied)
log.Printf("Error waiting for script %s: %v", s.scriptName, err)
s.exitCode = 1
}
}
processState, err := s.command.Process.Wait()
slog.Debug("Script exited", "script", s.scriptName, "error", err)
if !s.isRestarting {
s.wg.Done()
}
s.exitCode <- processState.ExitCode()
}()
return err
}
// it is the caller's responsibility to lock the mutex before calling this function
func (s *Script) Stop() error {
slog.Debug("Making sure process is still alive")
if runtime.GOOS == "windows" {
if _, err := os.FindProcess(s.command.Process.Pid); err != nil {
// process is already dead
return nil
}
} else {
process, err := os.FindProcess(s.command.Process.Pid)
if err != nil {
return err
}
// Sending signal 0 checks for existence and permissions
err = process.Signal(syscall.Signal(0))
if err != nil {
return nil
}
}
slog.Debug("Process is still alive, sending signal")
dead := make(chan bool)
go func() {
s.command.Wait()
dead <- true
}()
var signal syscall.Signal
switch s.zqdgr.Config.ShutdownSignal {
case "SIGINT":
signal = syscall.SIGINT
case "SIGTERM":
signal = syscall.SIGTERM
case "SIGQUIT":
signal = syscall.SIGQUIT
default:
signal = syscall.SIGKILL
}
if err := syscall.Kill(-s.command.Process.Pid, signal); err != nil {
log.Printf("error killing previous process: %v", err)
return err
}
shutdownTimeout := s.zqdgr.Config.ShutdownTimeout
if shutdownTimeout == 0 {
shutdownTimeout = 1
}
select {
case <-dead:
case <-time.After(time.Duration(shutdownTimeout) * time.Second):
log.Println("Script failed to stop after kill signal, force killing")
if err := syscall.Kill(-s.command.Process.Pid, syscall.SIGKILL); err != nil {
log.Printf("error killing previous process: %v", err)
return err
}
}
return nil
}
func (s *Script) Restart() error {
slog.Debug("Restarting script", "script", s.scriptName)
s.mutex.Lock()
s.isRestarting = true
if s.command.Process != nil {
var signal syscall.Signal
switch s.zqdgr.Config.ShutdownSignal {
case "SIGINT":
signal = syscall.SIGINT
case "SIGTERM":
signal = syscall.SIGTERM
case "SIGQUIT":
signal = syscall.SIGQUIT
default:
signal = syscall.SIGKILL
}
if err := syscall.Kill(-s.command.Process.Pid, signal); err != nil {
log.Printf("error killing previous process: %v", err)
}
err := s.Stop()
if err != nil {
s.mutex.Unlock()
return err
}
s.command = s.zqdgr.NewCommand(s.scriptName)
@@ -194,11 +239,9 @@ func (s *Script) Restart() error {
return nil
}
s.isRestarting = false
s.mutex.Unlock()
err := s.Start()
err = s.Start()
// tell the websocket clients to refresh
if s.zqdgr.EnableWebSocket {
@@ -217,10 +260,6 @@ func (s *Script) Restart() error {
return err
}
func (s *Script) Wait() {
s.wg.Wait()
}
func (wsServer *WSServer) handleWs(w http.ResponseWriter, r *http.Request) {
conn, err := wsServer.upgrader.Upgrade(w, r, nil)
if err != nil {
@@ -246,7 +285,10 @@ func (wsServer *WSServer) handleWs(w http.ResponseWriter, r *http.Request) {
func (zqdgr *ZQDGR) loadConfig() error {
data, err := os.ReadFile(path.Join(zqdgr.WorkingDirectory, "zqdgr.config.json"))
if err == nil {
if err := json.Unmarshal(data, &zqdgr.Config); err != nil {
decoder := json.NewDecoder(bytes.NewReader(data))
decoder.DisallowUnknownFields()
err := decoder.Decode(&zqdgr.Config)
if err != nil {
return fmt.Errorf("error parsing config file: %v", err)
}
} else {
@@ -262,24 +304,90 @@ func (zqdgr *ZQDGR) loadConfig() error {
return nil
}
func validatePattern(pattern string) ([]string, error) {
var paternArray []string
var currentPattern string
inMatch := false
// iterate over every letter in the pattern
for _, p := range pattern {
if string(p) == "{" {
if inMatch {
return nil, fmt.Errorf("unexpected { in pattern")
}
inMatch = true
}
if string(p) == "}" {
if !inMatch {
return nil, fmt.Errorf("enexpected } in pattern")
}
inMatch = false
}
if string(p) == "," && !inMatch {
paternArray = append(paternArray, currentPattern)
currentPattern = ""
continue
}
currentPattern += string(p)
}
if inMatch {
return nil, fmt.Errorf("unmatched } in pattern")
}
if currentPattern != "" {
paternArray = append(paternArray, currentPattern)
}
return paternArray, nil
}
func main() {
// var err error
var debugMode bool
noWs := flag.Bool("no-ws", false, "Disable WebSocket server")
configDir := flag.String("config", ".", "Path to the config directory")
flag.StringVar(configDir, "C", *configDir, "Path to the config directory")
disableReloadConfig := flag.Bool("no-reload-config", false, "Do not restart ZQDGR on config file change")
flag.Parse()
// if ParseBool returns an error, ZQDGR_DEBUG is not a bool
debugMode, _ = strconv.ParseBool(os.Getenv("ZQDGR_DEBUG"))
if debugMode {
slog.SetLogLoggerLevel(slog.LevelDebug)
}
originalArgs := os.Args
os.Args = flag.Args()
zqdgr := NewZQDGR(*noWs, *configDir)
expandedConfigDir, err := filepath.Abs(*configDir)
if err != nil {
log.Fatal(err)
}
slog.Debug("noWS", "noWs", *noWs, "configDir", *configDir, "expandedConfigDir", expandedConfigDir, "disableReloadConfig", *disableReloadConfig)
zqdgr := NewZQDGR(*noWs, expandedConfigDir)
if zqdgr == nil {
return
}
var command string
var commandArgs []string
// command name trimmed by flags.Args()
// os.Args ~= [script, --, arguments]
for i, arg := range os.Args {
if arg == "--" {
if i+2 < len(os.Args) {
commandArgs = os.Args[i+2:]
slog.Debug("Found double-dash", "i", i, "len(os.Args)", len(os.Args), "os.Args", os.Args)
if len(os.Args)-i > 1 {
commandArgs = os.Args[i+1:]
}
break
}
@@ -292,6 +400,8 @@ func main() {
commandArgs = append(commandArgs, arg)
}
slog.Debug("Collected", "command", command, "commandArgs", commandArgs)
watchMode := false
var scriptName string
switch command {
@@ -434,13 +544,7 @@ func main() {
log.Fatal("please specify a script to run")
}
watchMode = true
for i := range commandArgs {
if strings.HasPrefix(commandArgs[i], "-") {
continue
}
scriptName = commandArgs[i]
}
scriptName = commandArgs[0]
default:
scriptName = command
}
@@ -458,19 +562,9 @@ func main() {
log.Println("Received signal, exiting...")
if script.command != nil {
var signal syscall.Signal
switch zqdgr.Config.ShutdownSignal {
case "SIGINT":
signal = syscall.SIGINT
case "SIGTERM":
signal = syscall.SIGTERM
case "SIGQUIT":
signal = syscall.SIGQUIT
default:
signal = syscall.SIGKILL
}
syscall.Kill(-script.command.Process.Pid, signal)
script.mutex.Lock()
script.Stop()
script.mutex.Unlock()
}
os.Exit(0)
@@ -493,48 +587,22 @@ func main() {
log.Fatal("watch pattern not specified in config")
}
var paternArray []string
var currentPattern string
inMatch := false
// iterate over every letter in the pattern
for _, p := range zqdgr.Config.Pattern {
if string(p) == "{" {
if inMatch {
log.Fatal("unmatched { in pattern")
}
inMatch = true
}
if string(p) == "}" {
if !inMatch {
log.Fatal("unmatched } in pattern")
}
inMatch = false
}
if string(p) == "," && !inMatch {
paternArray = append(paternArray, currentPattern)
currentPattern = ""
inMatch = false
continue
}
currentPattern += string(p)
// make sure the pattern is valid
patternArray, err := validatePattern(zqdgr.Config.Pattern)
if err != nil {
log.Fatal(err)
}
if inMatch {
log.Fatal("unmatched } in pattern")
}
if currentPattern != "" {
paternArray = append(paternArray, currentPattern)
for _, pattern := range zqdgr.Config.ExcludedGlobs {
_, err := validatePattern(pattern)
if err != nil {
log.Fatal(err)
}
}
watcherConfig := WatcherConfig{
excludedDirs: globList(zqdgr.Config.ExcludedDirs),
pattern: paternArray,
excludedGlobs: globList(zqdgr.Config.ExcludedGlobs),
pattern: patternArray,
}
watcher, err := NewWatcher(&watcherConfig)
@@ -549,13 +617,17 @@ func main() {
log.Fatal(err)
}
// We use this timer to deduplicate events.
err = watcher.AddFile(path.Join(zqdgr.WorkingDirectory, "zqdgr.config.json"))
if err != nil {
log.Fatal(err)
}
// tailing edge debounce of file system events
var (
// Wait 100ms for new events; each new event resets the timer.
waitFor = 100 * time.Millisecond
// Keep track of the timers, as path → timer.
mu sync.Mutex
mu sync.Mutex
// watched filepath -> timer
timers = make(map[string]*time.Timer)
)
go func() {
@@ -572,10 +644,56 @@ func main() {
if !ok {
timer = time.AfterFunc(waitFor, func() {
slog.Debug("FSnotify event received", "event", event)
if event.Op&fsnotify.Remove == fsnotify.Remove || event.Op&fsnotify.Write == fsnotify.Write || event.Op&fsnotify.Create == fsnotify.Create {
fmt.Println("File changed:", event.Name)
if directoryShouldBeTracked(&watcherConfig, event.Name) {
watcher.(NotifyWatcher).watcher.Add(event.Name)
slog.Debug("File changed", "file", event.Name)
fullEventPath, err := filepath.Abs(event.Name)
if err != nil {
log.Fatal(err)
}
slog.Debug("expanded event path", "path", fullEventPath)
// check against the fullpath to make sure that the config file that was changed is
// actually the one we are using
if fullEventPath == filepath.Join(zqdgr.WorkingDirectory, "zqdgr.config.json") {
// re-exec the exact same command
if !*disableReloadConfig {
fmt.Println("zqdgr.config.json has changed, restarting...")
executable, err := os.Executable()
if err != nil {
log.Fatal(err)
}
script.mutex.Lock()
err = script.Stop()
if err != nil {
log.Fatal(err)
}
script.mutex.Unlock()
err = syscall.Exec(executable, originalArgs, os.Environ())
if err != nil {
log.Fatal(err)
}
panic("unreachable")
}
}
if pathShouldBeTracked(&watcherConfig, event.Name) {
if event.Op&fsnotify.Create == fsnotify.Create {
slog.Debug("Adding new file to watcher", "file", event.Name)
watcher.(NotifyWatcher).watcher.Add(event.Name)
}
if event.Op&fsnotify.Remove == fsnotify.Remove {
slog.Debug("Removing file from watcher", "file", event.Name)
watcher.(NotifyWatcher).watcher.Remove(event.Name)
}
}
if pathMatches(&watcherConfig, event.Name) {
@@ -608,8 +726,17 @@ func main() {
}
}
}()
// block until the script exits with a zero (aka, it normally exited on its own)
for {
exitCode := <-script.exitCode
slog.Debug("Script exited", "script", scriptName, "exitCode", exitCode)
if exitCode == 0 {
os.Exit(0)
}
}
}
script.Wait()
os.Exit(script.exitCode)
// block until the script exits
os.Exit(<-script.exitCode)
}

View File

@@ -4,7 +4,9 @@ import (
"errors"
"fmt"
"log"
"log/slog"
"path/filepath"
"strings"
"github.com/bmatcuk/doublestar"
"github.com/fsnotify/fsnotify"
@@ -14,27 +16,45 @@ type globList []string
func (g *globList) Matches(value string) bool {
for _, v := range *g {
// if the pattern matches a filepath pattern
if match, err := filepath.Match(v, value); err != nil {
log.Fatalf("Bad pattern \"%s\": %s", v, err.Error())
} else if match {
return true
}
// or if the path starts with the pattern
if strings.HasSuffix(value, v) {
return true
}
}
return false
}
func matchesPattern(pattern []string, path string) bool {
for _, p := range pattern {
slog.Debug("checking path against pattern", "pattern", p, "path", path)
if matched, _ := doublestar.Match(p, path); matched {
slog.Debug("path matches pattern", "pattern", p, "path", path)
return true
}
}
return false
}
func directoryShouldBeTracked(cfg *WatcherConfig, path string) bool {
func pathShouldBeTracked(cfg *WatcherConfig, path string) bool {
base := filepath.Dir(path)
return matchesPattern(cfg.pattern, path) && !cfg.excludedDirs.Matches(base)
slog.Debug("checking file against path", "path", path, "base", base)
if cfg.excludedGlobs.Matches(base) {
slog.Debug("file is excluded", "path", path)
return false
}
return matchesPattern(cfg.pattern, path)
}
func pathMatches(cfg *WatcherConfig, path string) bool {
@@ -42,13 +62,14 @@ func pathMatches(cfg *WatcherConfig, path string) bool {
}
type WatcherConfig struct {
excludedDirs globList
pattern globList
excludedGlobs globList
pattern globList
}
type FileWatcher interface {
Close() error
AddFiles() error
AddFile(path string) error
add(path string) error
getConfig() *WatcherConfig
}
@@ -62,6 +83,12 @@ func (n NotifyWatcher) Close() error {
return n.watcher.Close()
}
func (n NotifyWatcher) AddFile(path string) error {
slog.Debug("manually adding file", "file", path)
return n.add(path)
}
func (n NotifyWatcher) AddFiles() error {
return addFiles(n)
}
@@ -79,10 +106,12 @@ func NewWatcher(cfg *WatcherConfig) (FileWatcher, error) {
err := errors.New("no config specified")
return nil, err
}
w, err := fsnotify.NewWatcher()
if err != nil {
return nil, err
}
return NotifyWatcher{
watcher: w,
cfg: cfg,
@@ -92,18 +121,43 @@ func NewWatcher(cfg *WatcherConfig) (FileWatcher, error) {
func addFiles(fw FileWatcher) error {
cfg := fw.getConfig()
for _, pattern := range cfg.pattern {
slog.Debug("adding pattern", "pattern", pattern)
matches, err := doublestar.Glob(pattern)
if err != nil {
log.Fatalf("Bad pattern \"%s\": %s", pattern, err.Error())
}
trackedDirs := make(map[string]bool)
for _, match := range matches {
if directoryShouldBeTracked(cfg, match) {
base := filepath.Dir(match)
// this allows us to track file creations and deletions
if !trackedDirs[base] {
if cfg.excludedGlobs.Matches(base) {
slog.Debug("directory is excluded", "file", match)
continue
}
trackedDirs[base] = true
slog.Debug("adding directory", "dir", base)
if err := fw.add(base); err != nil {
return fmt.Errorf("FileWatcher.Add(): %v", err)
}
}
slog.Debug("adding file", "file", match)
if pathShouldBeTracked(cfg, match) {
slog.Debug("path should be tracked", "file", match)
if err := fw.add(match); err != nil {
return fmt.Errorf("FileWatcher.Add(): %v", err)
}
}
}
}
return nil
}

View File

@@ -1,6 +1,6 @@
{
"name": "zqdgr",
"version": "0.0.3",
"version": "0.0.6.1",
"description": "zqdgr is a quick and dirty Golang runner",
"author": "juls0730",
"license": "BSL-1.0",