From 0c62f7495ae9685b6478d5d5f9666a72f495873b Mon Sep 17 00:00:00 2001 From: WickedYoda <68271168+wickedyoda@users.noreply.github.com> Date: Fri, 4 Sep 2026 12:31:11 -0500 Subject: [PATCH] docs: add READMEs and wiki documentation for all directories - Add comprehensive wiki documentation under .github/wiki/ (19 articles) - Add READMEs to directories that lacked them - Fix merge conflict markers in existing READMEs - Update root readme.md with improved structure Co-authored-by: Jarvis --- .github/wiki/Core-Scripts.md | 148 ++++++++++ .github/wiki/Docker-Setup.md | 170 ++++++++++++ .github/wiki/Domain-Lists.md | 166 ++++++++++++ .github/wiki/Getting-Started.md | 91 +++++++ .github/wiki/Home.md | 103 +++++++ .github/wiki/Kali-Security.md | 120 +++++++++ .github/wiki/Linux-Setup.md | 184 +++++++++++++ .github/wiki/Monitoring.md | 214 +++++++++++++++ .github/wiki/NAS-Storage.md | 165 ++++++++++++ .github/wiki/Network.md | 196 ++++++++++++++ .github/wiki/OpenWrt.md | 155 +++++++++++ .github/wiki/Raspberry-Pi.md | 121 +++++++++ .github/wiki/System-Updates.md | 177 ++++++++++++ .github/wiki/Troubleshooting.md | 283 +++++++++++++++++++ .github/wiki/Windows-Batch-Files.md | 151 +++++++++++ .github/wiki/macOS-Scripts.md | 152 +++++++++++ Docker_backup/README.md | 82 +++++- Email_domains_block_allow/README.md | 81 +++--- batch_files/README.md | 85 +++--- copy-move-files/README.md | 95 ++++--- copy_palworld_local/README.md | 67 ++--- cron-job-setup-files/README.md | 93 ++++--- debian-dist-upgrades/README.md | 116 +++++--- debian-files/README.md | 85 +++--- docker/README.md | 72 +++++ domains/README.md | 93 ++++--- git_clone_setup/README.md | 78 +++--- kali-purple/README.md | 97 ++++--- paperless-ngx/README.md | 87 +++--- pihole/README.md | 84 ++++++ raspberrypi/README.md | 145 +++++++--- readme.md | 284 +++++++++++++++++--- redirecting-desktop_and_documents/README.md | 76 +++--- reset_perms/README.md | 90 ++++--- server_config/README.md | 123 ++++++--- snmp/README.md | 131 ++++++--- system_command_run/README.md | 114 +++++--- telegraf-setup-scripts/README.md | 102 ++++--- ubuntu-based/README.md | 98 ++++--- unblocked_domains_for_VPN/README.md | 91 ++++--- updates_scripts/README.md | 133 ++++++--- uptime-updates/README.md | 168 +++++++----- 42 files changed, 4501 insertions(+), 865 deletions(-) create mode 100644 .github/wiki/Core-Scripts.md create mode 100644 .github/wiki/Docker-Setup.md create mode 100644 .github/wiki/Domain-Lists.md create mode 100644 .github/wiki/Getting-Started.md create mode 100644 .github/wiki/Home.md create mode 100644 .github/wiki/Kali-Security.md create mode 100644 .github/wiki/Linux-Setup.md create mode 100644 .github/wiki/Monitoring.md create mode 100644 .github/wiki/NAS-Storage.md create mode 100644 .github/wiki/Network.md create mode 100644 .github/wiki/OpenWrt.md create mode 100644 .github/wiki/Raspberry-Pi.md create mode 100644 .github/wiki/System-Updates.md create mode 100644 .github/wiki/Troubleshooting.md create mode 100644 .github/wiki/Windows-Batch-Files.md create mode 100644 .github/wiki/macOS-Scripts.md create mode 100644 docker/README.md create mode 100644 pihole/README.md diff --git a/.github/wiki/Core-Scripts.md b/.github/wiki/Core-Scripts.md new file mode 100644 index 0000000..1e2bb61 --- /dev/null +++ b/.github/wiki/Core-Scripts.md @@ -0,0 +1,148 @@ +# Core Scripts + +This wiki documents the main automation scripts in the repository root. + +--- + +## `update-from_repo.sh` + +Bash script to clone/update the public-setupfiles repository. + +**Location:** Repository root + +**Usage:** +```bash +sudo ./update-from_repo.sh +``` + +**What it does:** +1. Auto-installs Git if not present (Debian/Ubuntu/OpenWrt) +2. Clones `https://github.com/wickedyoda/public-setupfiles.git` +3. Sets executable permissions on all files +4. Supports re-running to update existing clones + +**Options:** None (simple script) + +--- + +## `update-from_repo.py` + +Python version of the repository sync script. + +**Location:** Repository root + +**Usage:** +```bash +python3 update-from_repo.py +``` + +**Dependencies:** None (uses only stdlib) + +**What it does:** +- Same functionality as the shell script +- Installs Python 3 via apt if missing +- Uses subprocess to run git commands + +--- + +## `setup_cron_clean.sh` + +Installs automatic system updates as a system cron job. + +**Location:** Repository root + +**Usage:** +```bash +sudo ./setup_cron_clean.sh +``` + +**What it does:** +1. Installs cron package +2. Copies `updates_clean.sh` to `/usr/local/sbin/` +3. Creates `/etc/cron.d/auto_updates` +4. Schedules updates every 6 hours + +**Cron Schedule:** +| Task | Schedule | Command | +|------|----------|---------| +| System updates | `0 */6 * * *` | `/usr/local/sbin/updates_clean.sh` | +| Mount verification | `5 */1 * * *` | `mount -a` | + +**Log file:** `/var/log/auto_updates.log` + +**MAILTO:** Commented out by default. Uncomment and set your email if you want update notifications. + +--- + +## `updates_clean.sh` + +The actual update script called by cron. + +**Location:** Repository root (also copy at `/usr/local/sbin/updates_clean.sh`) + +**Run manually:** +```bash +sudo ./updates_clean.sh +``` + +**What it does:** +1. `apt update` — Refresh package lists +2. `apt full-upgrade -y` — Apply all updates +3. `apt autoremove -y` — Remove orphaned packages +4. `apt autopurge -y` — Purge config of removed packages (Debian 11+) +5. `apt clean -y` — Clear local cache + +**Logging:** All output goes to stdout and `/var/log/auto_updates.log` + +--- + +## `collect_info.sh` + +System information collection script. + +**Location:** Repository root + +**Usage:** +```bash +./collect_info.sh +``` + +**Output:** Creates `info.txt` with system details including: +- OS distribution and version +- Hostname and kernel +- CPU, memory, storage +- GPU and PCI/USB devices +- Network interfaces and routes +- Display configuration + +--- + +## `gather-logs/gather_logs.sh` + +Collects system logs for troubleshooting. + +**Location:** `gather-logs/` + +**Usage:** +```bash +./gather-logs.sh +``` + +**Generates:** `/tmp/{date}-{hostname}.zip` containing: +- `/var/log/` — syslog, auth, kern, daemon, dmesg, dpkg, apt +- `journalctl` — current boot and all logs +- System info — hostname, uname, df, free, ip addr, systemctl failed + +--- + +## Comparison: Shell vs Python Update Scripts + +| Feature | `update-from_repo.sh` | `update-from_repo.py` | +|---------|----------------------|----------------------| +| Language | Bash | Python | +| Git install | apt/opkg | apt only | +| Path typo | N/A | `pubic-setupfiles` (bug) | +| Permissions | 755 recursive | 777 recursive | +| Error handling | Basic | Try/except | + +**Note:** The Python script has a typo in the clone path (`pubic-setupfiles` instead of `public-setupfiles`). Prefer the shell script for now. \ No newline at end of file diff --git a/.github/wiki/Docker-Setup.md b/.github/wiki/Docker-Setup.md new file mode 100644 index 0000000..fde9e33 --- /dev/null +++ b/.github/wiki/Docker-Setup.md @@ -0,0 +1,170 @@ +# Docker Setup + +Docker configuration, installation scripts, and backup utilities. + +--- + +## Docker Installation Scripts + +### `docker/install_docker_debian.sh` + +Debian-based Docker Engine installation. + +**Usage:** +```bash +sudo ./install_docker_debian.sh +``` + +**Includes:** +- Docker Engine (docker-ce) +- Docker Compose plugin (v2) +- Optional: Adds current user to docker group + +**Post-install:** +```bash +# Log out and back in, or run: +newgrp docker +``` + +--- + +### `docker/install_docker_ubuntu.sh` + +Ubuntu-specific Docker installation. + +**Usage:** +```bash +sudo ./install_docker_ubuntu.sh +``` + +**Same functionality as debian script with Ubuntu package names.** + +--- + +## Docker Compose Examples + +### `docker/jellyfin-docker-compose.yml` + +Media server configuration. + +**Key settings:** +- Ports: 8096 (web), 8920 (HTTPS), 7359/1900 (UPnP) +- Volumes: config, cache, media +- Published URL: `http://docker2` + +**Usage:** +```bash +docker compose up -d +``` + +**Edit before running:** +- Update `volumes` paths to match your system +- Adjust `JELLYFIN_PublishedServerUrl` for your domain + +--- + +### `docker/portainer/portainers-installscript.sh` + +Portainer container installation. + +**Usage:** +```bash +sudo ./portainers-installscript.sh +``` + +**Creates:** +- Portainer data volume +- Portainer CE container on port 8099 (UI), 9443 (HTTPS) + +--- + +### `docker/portainer/update-portainer.sh` + +Update Portainer to latest version. + +**Usage:** +```bash +sudo ./update-portainer.sh +``` + +**What it does:** +1. Pulls `portainer/portainer-ce:latest` +2. Restarts the container + +--- + +## Docker Backup + +### Location: `Docker_backup/` + +**Scripts:** +| Script | Purpose | +|--------|---------| +| `docker-backup.sh` | Main backup script | +| `setup_docker-backup.sh` | Install backup + cron | +| `correct_docker-backup.sh` | Fix older installs | +| `update_docker-backup.sh` | Upgrade existing | + +**Backup Contents:** +``` +/mnt/naspublic/docker-backup/{hostname}/{month}/{date}/ +├── /root/docker +├── /opt/docker +├── /var/lib/docker/volumes +└── /home/traver/docker +``` + +**Schedule:** Every 3 hours + +**Retention:** 14 days + +**Usage:** +```bash +# First time setup +sudo ./setup_docker-backup.sh + +# Update existing install +sudo ./update_docker-backup.sh +``` + +--- + +## Docker Remote API + +### `docker_stuff/setup-docker-remote-api.sh` + +Enable Docker TCP API for remote monitoring (e.g., Uptime Kuma). + +**Usage:** +```bash +# Basic (insecure - listens on all interfaces) +sudo bash setup-docker-remote-api.sh + +# With IP restriction (secure) +sudo bash setup-docker-remote-api.sh --allow-ip 10.0.84.50 +``` + +**Warning:** This exposes the Docker API over unencrypted TCP. Only use on trusted networks with IP restrictions. + +**Default port:** 2375 + +--- + +## Docker Cleanup + +### `docker_stuff/docker_cleanup_updates.sh` + +Combined update and cleanup script. + +**What it does:** +1. Pulls setup scripts from GitHub +2. Runs system updates +3. Prunes unused Docker images +4. Prunes unused Docker networks +5. Removes stopped containers +6. Clears build cache + +**Usage:** +```bash +sudo ./docker_cleanup_updates.sh +``` \ No newline at end of file diff --git a/.github/wiki/Domain-Lists.md b/.github/wiki/Domain-Lists.md new file mode 100644 index 0000000..5163ad5 --- /dev/null +++ b/.github/wiki/Domain-Lists.md @@ -0,0 +1,166 @@ +# Domain Lists + +Domain blocklists, allowlists, and filtering rules used across the homelab. + +--- + +## Domain List Overview + +| Directory | Purpose | +|-----------|---------| +| `domains/` | General domain lists | +| `pihole/` | Pi-hole DNS blocklists | +| `unblocked_domains_for_VPN/` | VPN bypass domains | +| `Email_domains_block_allow/` | Email server policies | + +--- + +## Pi-hole Lists + +### Location: `pihole/` + +| File | Description | +|------|-------------| +| `blocklist.txt` | 80,000+ ad/malware/social media domains | +| `whitelist.txt` | Streaming services (Netflix, Hulu, etc.) | +| `combined_list.txt` | Merged block + allow lists | + +### Adding to Pi-hole + +```bash +# In Pi-hole web UI +Group Management → Adlists → Add + +# Use raw URLs: +https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/pihole/blocklist.txt +``` + +### Blocked Categories + +- **Advertising:** Google Ads, DoubleClick, Taboola, Outbrain +- **Social Media:** Facebook, Instagram, Twitter, TikTok, LinkedIn +- **Adult Content:** Pornhub, XVideos, RedTube, YouPorn +- **Malware:** Known malicious domains +- **Gambling:** Betting and casino sites +- **Gaming:** Roblox, Discord, gaming trackers + +--- + +## Unblocked VPN Domains + +### Location: `unblocked_domains_for_VPN/` + +**Purpose:** Domains that should bypass the VPN connection. + +**File:** `unblocked_domains.txt` + +**Includes:** +- Personal domains: `wickedyoda.com`, `tyates.one`, `twy4.us` +- Work domains: `traverates.com`, `sampyates.com`, `designsbymantha.com` +- Service domains: Tailscale, Home Assistant, etc. + +--- + +## General Domain Lists + +### Location: `domains/` + +#### Social Media + +**File:** `domains/social_media.txt` + +Major platforms tracked: +- Facebook/Meta family +- Instagram +- Twitter/X +- TikTok +- LinkedIn +- Snapchat +- Threads +- Bluesky + +--- + +#### Streaming Services + +**File:** `domains/streaming_domains_whitelist.txt` + +**Included:** +- Netflix (and CDN) +- Hulu +- HBO Max +- Disney+ +- Paramount+ +- Peacock +- Amazon Prime Video + +--- + +#### Adult Content + +**File:** `domains/known_porn_domains.txt` + +Large blocklist of adult content domains. + +--- + +#### Country Domains + +**File:** `domains/sweden_domains.txt` + +Domains associated with Sweden (for geo-filtering). + +--- + +## Email Domain Filtering + +### Location: `Email_domains_block_allow/` + +| File | Purpose | +|------|---------| +| `blocked_domains.md` | Domains to reject in SMTP | +| `email_domain_allow_block_list.md` | Comprehensive rules | + +### Use Cases + +- **Postfix:** `header_checks`, `sender_access` +- **Exim:** `acl_smtp_dnsbl` +- **Exchange:** Transport Rules + +--- + +## Bambu Lab Domains + +### Location: `domains/bamboo_domains.txt` + +Domains for Bambu Lab 3D printers: + +``` +bambulab.com +api.bambulab.com +public-cdn.bambulab.cn +cloud.bambulab.com +ota.bambulab.com +mqtt.bambulab.com +``` + +--- + +## Usage with Other Tools + +### Pi-hole Adlist Format + +``` +https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/pihole/blocklist.txt +``` + +### Hosts File Format + +``` +127.0.0.1 domain.com +::1 domain.com +``` + +### BIND/DNS Server + +Import as zone files or use as external lookup sources. \ No newline at end of file diff --git a/.github/wiki/Getting-Started.md b/.github/wiki/Getting-Started.md new file mode 100644 index 0000000..b0a05e9 --- /dev/null +++ b/.github/wiki/Getting-Started.md @@ -0,0 +1,91 @@ +# Getting Started + +This guide helps you get started with the **public-setupfiles** repository. + +## 📥 Cloning + +### Quick Install (Recommended) + +The fastest way to get started is using the provided update script: + +```bash +sudo curl -s https://raw.githubusercontent.com/wickedyoda/public-setupfiles/refs/heads/main/update-from_repo.sh | sudo bash +``` + +This script: +- Automatically installs Git if missing +- Clones the repository to `./public-setupfiles` +- Sets proper permissions + +### Manual Clone + +```bash +git clone https://github.com/wickedyoda/public-setupfiles.git +cd public-setupfiles +chmod -R 755 . +``` + +### Python Version (Alternative) + +```bash +curl -s https://raw.githubusercontent.com/wickedyoda/public-setupfiles/refs/heads/main/update-from_repo.py | python3 +``` + +--- + +## 🔄 Keeping Updated + +### Automated Updates + +Set up the automated update cron job: + +```bash +sudo ./setup_cron_clean.sh +``` + +This installs `updates_clean.sh` to run every 6 hours. + +### Manual Update + +```bash +# From the repo root +./update-from_repo.sh + +# Or using Python +python3 update-from_repo.py +``` + +--- + +## 📁 Navigation + +| Directory | Purpose | Key Scripts | +|-----------|---------|-------------| +| `docker/` | Docker installations | `install_docker_debian.sh` | +| `server_config/` | Server setup templates | `debian-setup.sh` | +| `uptime-updates/` | Monitoring automation | `uptime-bot.py` | +| `snmp/` | Network monitoring | `setup_snmp.sh` | +| `pihole/` | DNS blocking | `blocklist.txt` | +| `domains/` | Domain lists | `blocked_domains.md` | + +--- + +## ⚠️ Safety Notes + +- Always review scripts before running them +- Some scripts require `sudo` or root access +- Test in a non-production environment first +- Backup critical data before making changes + +--- + +## 🔧 Tools Used + +| Tool | Purpose | +|------|---------| +| **ShellCheck** | Shell script linting | +| **CodeQL** | Static code analysis | +| **pip-audit** | Dependency vulnerability scan | +| **Docker Scout** | Container image security | + +See [Security Scanning](Security-Scanning.md) for details. \ No newline at end of file diff --git a/.github/wiki/Home.md b/.github/wiki/Home.md new file mode 100644 index 0000000..4c555ef --- /dev/null +++ b/.github/wiki/Home.md @@ -0,0 +1,103 @@ +# Public Setup Files — Repository Wiki + +Welcome to the **public-setupfiles** wiki! This repository contains a collection of scripts and configuration snippets used across various systems in the WickedYoda homelab. + +--- + +## 📚 Wiki Index + +| Category | Description | +|---------|-------------| +| [Getting Started](Getting-Started.md) | Quick start guide for cloning and using the repository | +| [Core Scripts](Core-Scripts.md) | Overview of `update-from-repo` and `setup_cron_clean.sh` | +| [Docker Setup](Docker-Setup.md) | Docker installation and backup utilities | +| [System Updates](System-Updates.md) | Automated update and cleanup workflows | +| [Linux Setup](Linux-Setup.md) | Debian and Ubuntu system configuration scripts | +| [Monitoring](Monitoring.md) | Uptime Kuma bot, SNMP, Telegraf, Observium | +| [Network](Network.md) | WireGuard VPN, Pi-hole, domains blocklists | +| [NAS & Storage](NAS-Storage.md) | fstab setup, backup scripts | +| [Windows Batch Files](Windows-Batch-Files.md) | Windows automation scripts | +| [macOS Scripts](macOS-Scripts.md) | Homebrew and system update utilities | +| [Raspberry Pi](Raspberry-Pi.md) | Raspberry Pi specific setup scripts | +| [OpenWrt](OpenWrt.md) | Router configuration utilities | +| [Kali & Security](Kali-Security.md) | Kali Linux and security tool installation | +| [Domain Lists](Domain-Lists.md) | Email, VPN, and content domain lists | +| [Troubleshooting](Troubleshooting.md) | Common issues and solutions | + +--- + +## 🚀 Cloning the Repository + +```bash +# Clone and set up in one command +sudo curl -s https://raw.githubusercontent.com/wickedyoda/public-setupfiles/refs/heads/main/update-from_repo.sh | sudo bash +``` + +--- + +## 📁 Repository Structure + +``` +public-setupfiles/ +├── .github/ # GitHub workflows and security config +│ ├── workflows/ +│ └── SECURITY_SCANS.md +├── Docker_backup/ # Docker data backup scripts +├── Email_domains_block_allow/ # Email filtering domain lists +├── Kali_tools_install/ # Kali Linux security tools +├── WG VPN setup/ # WireGuard client setup +├── batch_files/ # Windows batch automation +├── cockpit/ # Cockpit web admin installer +├── copy-move-files/ # Media file transfer scripts +├── copy_palworld_local/ # Palworld server backup +├── cron-job-setup-files/ # Cron job configuration +├── debian-dist-upgrades/ # Debian version upgrade scripts +├── debian-files/ # Debian utility scripts +├── docker/ # Docker compose setups +├── docker_stuff/ # Docker cleanup scripts +├── domains/ # Domain block/allow lists +├── fstab-setup/ # NFS/CIFS mount configuration +├── gather-logs/ # System diagnostic script +├── git_clone_setup/ # Git repository helpers +├── kali-purple/ # Kali Purple installation +├── mac-scripts/ # macOS automation +├── observium/ # Observium monitoring agent +├── openwrt_scripts/ # OpenWrt router utilities +├── paperless-ngx/ # Document management setup +├── pihole/ # Pi-hole blocklists +├── raspberrypi/ # Raspberry Pi setup scripts +├── redirecting-desktop_and_documents/ # Windows folder redirection +├── reset_perms/ # Permission reset utilities +├── server_config/ # Debian/Ubuntu server templates +├── snmp/ # SNMP configuration scripts +├── system_command_run/ # Distributed command execution +├── telegraf-setup-scripts/ # Telegraf metrics agent +├── ubuntu-based/ # Ubuntu application installer +├── unblocked_domains_for_VPN/ # VPN domain whitelist +├── updates_scripts/ # System update automation +└── uptime-updates/ # Uptime Kuma container sync bot +``` + +--- + +## 🛡️ Security + +- This repository undergoes automated security scanning on every pull request. +- Review [SECURITY.md](SECURITY.md) for details on the scanning tools used. +- **Never commit secrets, credentials, or private keys.** + +--- + +## 🤝 Contributing + +1. Fork the repository +2. Create a feature branch +3. Submit a pull request + +All PRs are automatically scanned for security issues. + +--- + +## 📞 Contact + +For issues or questions, open an issue on GitHub or contact the maintainer through the WickedYoda homelab community. \ No newline at end of file diff --git a/.github/wiki/Kali-Security.md b/.github/wiki/Kali-Security.md new file mode 100644 index 0000000..2da9c1e --- /dev/null +++ b/.github/wiki/Kali-Security.md @@ -0,0 +1,120 @@ +# Kali Linux & Security Tools + +Kali Linux setup and security tool installation scripts. + +--- + +## Kali Tools Installation + +### Location: `Kali_tools_install/` + +| File | Purpose | +|------|---------| +| `setup_debian_kali_tools.sh` | Install Kali tools on Debian | +| `Kalitoolsinstall.sh` | Direct Kali tools install | + +--- + +### `setup_debian_kali_tools.sh` + +Install Kali security tools on a Debian system. + +**Usage:** +```bash +sudo ./Kali_tools_install/setup_debian_kali_tools.sh +``` + +**Tool Categories:** + +| Category | Tools | +|----------|-------| +| Identify | `kali-tools-identify` | +| Protect | `kali-tools-protect` | +| Detect | `kali-tools-detect` | +| Respond | `kali-tools-respond` | +| Recover | `kali-tools-recover` | + +--- + +## Kali Purple + +### Location: `kali-purple/` + +Kali Linux Purple security platform setup. + +### Files + +| File | Purpose | +|------|---------| +| `install_kali_purple.sh` | Install Kali Purple tools | + +--- + +### Installation + +```bash +sudo ./kali-purple/install_kali_purple.sh +``` + +**What it installs:** +1. Kali Rolling sources +2. System updates +3. Kali Purple tools (`kali-tools-identify`, `kali-tools-protect`, etc.) +4. Purple theme (`kali-themes-purple`) +5. Legacy wallpapers + +**Interactive prompts:** +- Kali tool categories (1-5 or 'all') +- Purple theme options (1-4) + +--- + +## Kali Linux Conversion + +### `debian-dist-upgrades/dist-upgrade.sh` + +Option 4 allows converting Debian to Kali Linux. + +```bash +# From the dist-upgrade script +4) Convert Debian to Parrot Linux + +# For Kali, modify or use: +git clone https://gitlab.com/parrotsec/project/debian-conversion-script.git +cd debian-conversion-script +chmod +x ./install.sh +./install.sh +``` + +--- + +## Security Tool References + +### Common Tools + +| Category | Tools | +|----------|-------| +| Network Scanning | nmap, masscan | +| Web Apps | nikto, sqlmap, wfuzz | +| Wireless | aircrack-ng, reaver | +| Forensics | autopsy, volatility | +| Exploitation | metasploit, exploitdb | +| Password | john, hashcat, hydra | + +--- + +## Usage Recommendations + +### Virtual Environment + +```bash +# Never run on production systems without testing +# Use VirtualBox or VMware for isolated testing +``` + +### Legal Compliance + +```bash +# Only scan networks you own or have permission to test +# This software is for authorized security testing only +``` \ No newline at end of file diff --git a/.github/wiki/Linux-Setup.md b/.github/wiki/Linux-Setup.md new file mode 100644 index 0000000..48abd8c --- /dev/null +++ b/.github/wiki/Linux-Setup.md @@ -0,0 +1,184 @@ +# Linux Setup + +Debian and Ubuntu system configuration scripts. + +--- + +## Debian Files + +### `debian-files/` + +| Script | Purpose | +|--------|---------| +| `install-bashtop.sh` | Install system monitoring tool | +| `update_tailscale.sh` | Update Tailscale with keyring | + +#### Bashtop Installation + +```bash +sudo ./debian-files/install-bashtop.sh +``` + +Installs from GitHub source (latest development version). + +#### Tailscale Update + +```bash +sudo ./debian-files/update_tailscale.sh +``` + +**Features:** +- Detects Debian/Ubuntu/Raspbian +- Normalizes codenames (bullseye, bookworm, etc.) +- Falls back to legacy `apt-key` for older releases +- Installs from official Tailscale repo + +--- + +## Debian Distro Upgrades + +### `debian-dist-upgrades/` + +| File | Purpose | +|------|---------| +| `README.md` | This documentation | +| `dist-upgrade.sh` | Interactive upgrade script | + +#### `dist-upgrade.sh` + +```bash +sudo ./debian-dist-upgrades/dist-upgrade.sh +``` + +**Supported upgrades:** +- Debian 11 → 12 (stable) +- Debian 12 → 13 (testing) + +**Options:** +``` +1) apt upgrade (security + bug fixes) +2) apt full-upgrade (recommended) +3) Debian distro upgrade (major version) +4) Convert to Parrot Linux (security-focused) +``` + +**Safety features:** +- Backup of `/etc/apt/sources.list` +- Double confirmation for Parrot conversion +- Automatic dependency handling + +--- + +## Server Configuration + +### `server_config/` + +#### `server_config/debian/` + +| File | Purpose | +|------|---------| +| `debian-setup.sh` | Initial Debian server setup | + +**Commands included:** +```bash +# Install basic tools +apt install -y wireshark nmap openvpn etherape remmina monitorix + +# Install networking +apt install -y openssh-server smbclient cifs-utils + +# Install utilities +apt install -y curl unattended-upgrades cron-apt git +``` + +#### `server_config/ubuntu/` + +| File | Purpose | +|------|---------| +| `ubuntu-setup.sh` | Ubuntu server initial setup | +| `ubuntu-docker-install.sh` | Docker on Ubuntu | + +--- + +## fstab Setup + +### `fstab-setup/` + +| File | Purpose | +|------|---------| +| `Fstab entries.txt` | NAS mount configuration examples | +| `setup-fstab.sh` | Generate and apply fstab entries | +| `setup-fstab-remake-mnt.sh` | Reset and remount | + +#### NAS Mount Entries + +```bash +# Public share +//nas/public /media/naspublic cifs vers=2.0,username=admin,password=YOURPASS,file_mode=0777,dir_mode=0777,_netdev,auto 0 0 + +# Media share +//nas/DownloadedMedia /media/nasmedia cifs vers=2.0,username=admin,password=YOURPASS,file_mode=0777,dir_mode=0777,_netdev,auto 0 0 +``` + +#### Setup Script + +```bash +sudo ./fstab-setup/setup-fstab.sh +``` + +**What it does:** +1. Backs up existing `/etc/fstab` +2. Creates mount points +3. Appends new entries +4. Runs `mount -a` to activate + +--- + +## Reset Permissions + +### `reset_perms/` + +| File | Purpose | +|------|---------| +| `reset_perms.sh` | Reset permissions on mount points | +| `reset_perms 1.sh` | Alternative version | + +#### Usage + +```bash +sudo ./reset_perms/reset_perms.sh +``` + +**Resets:** +```bash +/mnt/public /dev/sdc1 +/mnt/public-bk /dev/sde1 +/mnt/public2 /dev/sdd1 +/mnt/public2-bk /dev/sdb1 +/mnt/timeshift bind mount from public2 +/mnt/timeshift1 bind mount from public2-bk +``` + +**Sets:** +- Ownership: `root:root` +- Permissions: `777` (world read/write/execute) + +--- + +## Gather Logs + +### `gather-logs/` + +**Script:** `gather_logs.sh` + +```bash +./gather-logs/gather_logs.sh +``` + +**Collects:** +- `/var/log/syslog`, `/var/log/auth.log`, `/var/log/kern.log` +- `journalctl` output (current boot + all) +- System info: hostname, uname, df, free, ip addr +- Failed systemd units + +**Output:** `/tmp/{date}-{hostname}.zip` \ No newline at end of file diff --git a/.github/wiki/Monitoring.md b/.github/wiki/Monitoring.md new file mode 100644 index 0000000..7236de7 --- /dev/null +++ b/.github/wiki/Monitoring.md @@ -0,0 +1,214 @@ +# Monitoring + +Uptime Kuma automation, SNMP, Telegraf, and Observium monitoring tools. + +--- + +## Uptime Updates Bot + +### Location: `uptime-updates/` + +A Python bot that syncs Docker container IDs with Uptime Kuma monitors when containers are recreated. + +### Files + +| File | Purpose | +|------|---------| +| `uptime-bot.py` | Main bot script | +| `config.yml` | Uptime Kuma API config | +| `container-monitor-map.yaml` | Container-to-monitor mappings | +| `requirements.txt` | Python dependencies | +| `setup-update_uptime-bot.sh` | Install script | +| `export-watchtower-bundle.sh` | Export Docker bundles | + +### Configuration + +#### `config.yml` + +```yaml +uptime_kuma: + url: "http://127.0.0.1:3001" + api_key: "PUT_YOUR_API_KEY_HERE" + +bot: + dry_run: false + request_timeout_seconds: 10 + poll_interval_seconds: 60 +``` + +#### `container-monitor-map.yaml` + +```yaml +mappings: + - name: "media-server jellyfin" + host_api: "tcp://10.0.84.21:2375" + container_name: "jellyfin" + uptime_monitor_id: 21 + expected_uptime_type: "docker" +``` + +### Installation + +```bash +sudo bash setup-update_uptime-bot.sh +``` + +**Options:** +```bash +sudo bash setup-update_uptime-bot.sh --help + +Options: + --install-dir PATH Install directory (default: /opt/uptime-updates) + --service-name NAME systemd unit name + --user USER User to run as + --group GROUP Group to run as + --no-enable Skip enabling on boot + --no-start Skip starting after install + --overwrite-config Replace existing config + --overwrite-map Replace existing map + --dry-run Print actions without executing +``` + +### Standalone Docker Deployment + +#### `watchtower-export/` + +Includes Docker Compose files for running without systemd: + +- `docker-compose.watchtower-uptime-sync.yml` — Separate containers +- `docker-compose.watchtower-embedded.yml` — Embedded in watchtower + +**Run:** +```bash +docker compose -f docker-compose.watchtower-uptime-sync.yml up -d +``` + +--- + +## SNMP Configuration + +### Location: `snmp/` + +SNMP setup scripts for various platforms. + +#### Linux + +**Files:** +| Script | Purpose | +|--------|---------| +| `setup_snmp.sh` | Standard Linux SNMPv3 setup | +| `setup_snmp_openwrt.sh` | OpenWrt SNMPv3 | +| `setup_snmp_openwrt_v2.sh` | OpenWrt SNMPv3 (alternative) | +| `setup_snmp_openwrt_publicv2.sh` | OpenWrt SNMPv2 (public) | + +**Usage (Linux):** +```bash +sudo ./snmp/setup_snmp.sh +``` + +**Prompts:** +- SNMPv3 username and password +- System location +- System contact + +**Features:** +- Creates `createUser` entry +- Sets up `rouser` with authPriv +- Configures firewall rules + +#### macOS + +**File:** `snmp/Setup_snmp_macos.sh` + +```bash +brew install net-snmp +sudo ./snmp/Setup_snmp_macos.sh +``` + +--- + +## Telegraf Setup + +### Location: `telegraf-setup-scripts/` + +| File | Purpose | +|------|---------| +| `install-client.sh` | Install Telegraf client | +| `update-telegraf-w-keys.sh` | Update with SSH keys | + +**Usage:** +```bash +curl -s https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/telegraf-setup-scripts/install-client.sh | sudo bash +``` + +--- + +## Observium Agent + +### Location: `observium/` + +Open-source network monitoring platform agent installer. + +**Setup:** +```bash +sudo wget https://raw.githubusercontent.com/tkrause/Observium-Agent/master/agent.conf.sh +sudo wget https://raw.githubusercontent.com/tkrause/Observium-Agent/master/observium-agent-install.sh +sudo chmod +x agent.conf.sh observium-agent-install.sh +sudo ./observium-agent-install.sh +``` + +**Agent Config Options:** +``` +$SYSCONTACT # Email: "Name " +$SYSLOCATION # Physical location +$SNMP_COMMUNITY # SNMP community string +$OBSERVIUM_HOST # Observium server IP +$MODULES # Optional: space-separated modules +$SVN_USER # Enterprise license username +$SVN_PASS # Enterprise license password +``` + +--- + +## Pi-hole + +### Location: `pihole/` + +Blocklists for Pi-hole DNS sinkhole. + +| File | Type | +|------|------| +| `blocklist.txt` | Domains to block (adult, porn, gambling, etc.) | +| `whitelist.txt` | Netflix streaming domains | +| `combined_list.txt` | Merged block + allow lists | + +**Usage:** +```bash +# In Pi-hole web UI +# Group Management → Adlists +# Add: https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/pihole/blocklist.txt +``` + +**Estimated domains blocked:** +- 80,000+ ad/malware domains +- 100+ social media domains +- 100+ streaming domains (whitelist) + +--- + +## Domain Lists + +### Location: `domains/` + +| Directory | Description | +|-----------|-------------| +| `Email_domains_block_allow/` | Email server domain policies | +| `unblocked_domains_for_VPN/` | Domains allowed through VPN | + +**Email Domain Files:** +- `blocked_domains.md` — Domains to block for email +- `email_domain_allow_block_list.md` — Allow/block rules + +**Unblocked Domains:** +- Personal domains (wickedyoda.com, tyates.one) +- Work domains (traveryates.com, sampyates.com, etc.) \ No newline at end of file diff --git a/.github/wiki/NAS-Storage.md b/.github/wiki/NAS-Storage.md new file mode 100644 index 0000000..8c4fe39 --- /dev/null +++ b/.github/wiki/NAS-Storage.md @@ -0,0 +1,165 @@ +# NAS & Storage + +Network Attached Storage setup, SMB/CIFS mounting, and backup scripts. + +--- + +## SMB/NAS Mount Setup + +### `fstab-setup/` + +Configure NAS shares via `/etc/fstab`. + +| File | Purpose | +|------|---------| +| `Fstab entries.txt` | Pre-built mount examples | +| `setup-fstab.sh` | Apply fstab changes | +| `setup-fstab-remake-mnt.sh` | Re-create mount points | + +### Mount Entry Format + +``` +//server/share /mount/point cifs options 0 0 +``` + +### Common Entries + +```bash +# Public share (read/write) +//nas/public /mnt/naspublic cifs vers=2.0,username=admin,password=YOURPASS,file_mode=0777,dir_mode=0777,_netdev,auto 0 0 + +# Download media +//nas/DownloadedMedia /mnt/nasmedia cifs vers=2.0,username=admin,password=YOURPASS,file_mode=0777,dir_mode=0777,_netdev,auto 0 0 + +# Public2 share +//nas/public-share2 /mnt/naspublic-share2 cifs vers=2.0,username=admin,password=YOURPASS,file_mode=0777,dir_mode=0777,_netdev,auto 0 0 +``` + +### Setup Script + +```bash +sudo ./fstab-setup/setup-fstab.sh +``` + +--- + +## NAS Share Management + +### `copy-move-files/` + +Scripts for copying and moving media between NAS and local storage. + +| Script | Purpose | +|--------|---------| +| `media_to_usb.sh` | Copy NAS media to USB drive | +| `public_to_usb.sh` | Transfer public share to USB | +| `nas public.sh` | Sync from NAS to local | +| `Brandons-Minecraft_backup.sh` | Minecraft world backup | + +### Example Usage + +```bash +# Copy media to USB +sudo ./copy-move-files/media_to_usb.sh + +# Sync public share +sudo ./copy-move-files/nas\ public.sh +``` + +--- + +## Docker Backup + +### Location: `Docker_backup/` + +Complete Docker data backup solution. + +| File | Purpose | +|------|---------| +| `docker-backup.sh` | Main backup script | +| `setup_docker-backup.sh` | Install + schedule | +| `correct_docker-backup.sh` | Fix older installs | +| `update_docker-backup.sh` | Upgrade existing | + +### Backup Contents + +``` +/mnt/naspublic/docker-backup/{hostname}/{month}/{date}/ +├── /root/docker +├── /opt/docker +├── /var/lib/docker/volumes +└── /home/traver/docker +``` + +### Installation + +```bash +sudo ./Docker_backup/setup_docker-backup.sh +``` + +**Schedule:** Every 3 hours via cron + +**Retention:** 14 days + +--- + +## Raspberry Pi Storage + +### Location: `raspberrypi/` + +SD card and USB storage management. + +| Script | Purpose | +|--------|---------| +| `raspberry_pi_fstab.sh` | Configure fstab for Pi | +| `rasp_pi_install_apps.sh` | Install apps on Pi | +| `Media_to_ap-share.sh` | Sync to AP share | +| `public_to_usb.sh` | Transfer to USB | + +--- + +## Permission Management + +### `reset_perms/` + +Reset file permissions on mount points. + +```bash +sudo ./reset_perms/reset_perms.sh +``` + +**Resets permissions to:** +- Owner: `root:root` +- Permissions: `777` (full read/write/execute) + +**Applies to:** +``` +/mnt/public +/mnt/public-bk +/mnt/public2 +/mnt/public2-bk +/mnt/timeshift +/mnt/timeshift1 +``` + +--- + +## System Information + +### `collect_info.sh` (repo root) + +Generate system information report. + +```bash +./collect_info.sh +``` + +**Output:** `info.txt` + +**Contains:** +- OS and kernel version +- CPU and memory +- Storage layout +- Network interfaces +- GPU information +- Display configuration \ No newline at end of file diff --git a/.github/wiki/Network.md b/.github/wiki/Network.md new file mode 100644 index 0000000..14e9d72 --- /dev/null +++ b/.github/wiki/Network.md @@ -0,0 +1,196 @@ +# Network + +VPN, firewall, domain lists, and network automation scripts. + +--- + +## WireGuard VPN + +### Location: `WG VPN setup/` + +WireGuard client tunnel setup for connecting to the Flint4 router. + +### Installation + +```bash +# Download installer +curl -sSL https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/WG%20VPN%20setup/install-wireguard-client.sh -o install-wireguard-client.sh +chmod +x install-wireguard-client.sh + +# Run +sudo ./install-wireguard-client.sh +``` + +### What the Script Does + +1. **Installs WireGuard:** `wireguard` and `wireguard-tools` packages +2. **Copies config:** Places it at `/etc/wireguard/wgclient2.conf` +3. **Creates systemd service:** `wg-wgclient2.service` +4. **Activates tunnel:** Brings up the interface + +### Management Commands + +```bash +# Check tunnel status +wg show wgclient2 + +# Restart tunnel +sudo systemctl restart wg-wgclient2 + +# View logs +sudo journalctl -u wg-wgclient2 -f +``` + +### Configuration + +Default settings: +- **Tunnel name:** `wgclient2` +- **Peer host:** `twy4us.duckdns.org` (Flint4 DuckDNS) +- **Config location:** `/etc/wireguard/wgclient2.conf` + +**To customize:** Edit `install-wireguard-client.sh` before running. + +--- + +## OpenVPN + +### `ubuntu-based/ubuntu_install_apps.sh` + +Includes OpenVPN client installation: + +```bash +sudo apt update +sudo apt install -y openvpn +``` + +**Usage:** +```bash +sudo openvpn --config my-vpn.ovpn +``` + +--- + +## Pi-hole DNS + +### Location: `pihole/` + +DNS-based ad blocking and malware protection. + +### Blocklists + +| File | Description | +|------|-------------| +| `blocklist.txt` | 80,000+ ad/malware/social media domains | +| `whitelist.txt` | Netflix streaming domains | +| `combined_list.txt` | Merged block + allow lists | + +### Integration + +**Add to Pi-hole:** +```bash +# Via web UI +Group Management → Adlists → Add + +# URL +https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/pihole/blocklist.txt +``` + +**Categories blocked:** +- Ad networks (Google Ads, DoubleClick, etc.) +- Social media trackers +- Adult content +- Gambling sites +- Malware domains + +--- + +## Domain Lists + +### `unblocked_domains_for_VPN/` + +Domains that should bypass VPN (allow direct connection). + +**File:** `unblocked_domains.txt` + +**Contents:** +- Personal sites (wickedyoda.com, tyates.one) +- Work/business domains +- Local network domains + +--- + +### `domains/` + +Various domain lists for different purposes. + +| File | Use Case | +|------|----------| +| `social_media.txt` | Social networking sites | +| `streaming_domains_whitelist.txt` | Netflix, Hulu, HBO Max, etc. | +| `known_porn_domains.txt` | Adult content | +| `bamboo_domains.txt` | Bambu Lab printer domains | + +--- + +## Email Domain Filtering + +### `Email_domains_block_allow/` + +Domain policies for email servers. + +| File | Purpose | +|------|---------| +| `blocked_domains.md` | Domains to reject at SMTP level | +| `email_domain_allow_block_list.md` | Comprehensive allow/block rules | + +**Use with:** +- Postfix +- Exim +- Microsoft Exchange +- Any SMTP server supporting domain-based filtering + +--- + +## Network Tools + +### `docker_stuff/` + +**File:** `docker_cleanup_updates.sh` + +Combined Docker and system update script with network verification. + +**Features:** +- System package updates +- Docker image/network pruning +- Container cleanup +- Build cache removal + +### `docker/install_docker_debian.sh` + +Docker installation with proper networking: +```bash +# After install, add user to docker group +sudo usermod -aG docker $USER +newgrp docker # Or log out/in +``` + +--- + +## OpenWrt Router Scripts + +### Location: `openwrt_scripts/` + +| File | Purpose | +|------|---------| +| `openwrt_full-upgrade.sh` | Full OpenWrt firmware upgrade | +| `installing_ipref.sh` | IP configuration scripts | +| `openwrt_snmp_defaults.txt` | SNMP configuration templates | + +**Usage:** +```bash +# SSH to router +ssh root@10.0.84.1 + +# Download and run +curl -sSL https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/openwrt_scripts/openwrt_full-upgrade.sh | sh +``` \ No newline at end of file diff --git a/.github/wiki/OpenWrt.md b/.github/wiki/OpenWrt.md new file mode 100644 index 0000000..57d8447 --- /dev/null +++ b/.github/wiki/OpenWrt.md @@ -0,0 +1,155 @@ +# OpenWrt Router Setup + +OpenWrt router configuration and management scripts. + +--- + +## Scripts + +### Location: `openwrt_scripts/` + +| File | Purpose | +|------|---------| +| `openwrt_full-upgrade.sh` | Full firmware upgrade | +| `installing_ipref.sh` | IP configuration | +| `openwrt_snmp_defaults.txt` | SNMP config template | + +--- + +## Full System Upgrade + +### `openwrt_full-upgrade.sh` + +Perform a complete OpenWrt upgrade. + +**Usage:** +```bash +ssh root@10.0.84.1 +curl -sSL https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/openwrt_scripts/openwrt_full-upgrade.sh | sh +``` + +**Process:** +1. Downloads latest firmware +2. Validates checksum +3. Backs up config +4. Performs sysupgrade + +--- + +## IP Configuration + +### `installing_ipref.sh` + +Configure IP addresses and interfaces. + +**Typical usage:** +```bash +# Access router via SSH +ssh root@192.168.1.1 + +# Run configuration script +./installing_ipref.sh +``` + +--- + +## SNMP on OpenWrt + +### `openwrt_snmp_defaults.txt` + +Default SNMP configuration for OpenWrt routers. + +```bash +# Edit /etc/config/snmpd +config agent + option agentaddress 'udp:161' + +config view + option name 'all' + option type 'included' + option oid '.1' + +config group + option groupname 'publicGroup' + option securitymodel 'v2c' + +config access + option groupname 'publicGroup' + option version 'v2c' + option level 'noAuthNoPriv' +``` + +### Installation + +```bash +# Install SNMP +opkg update +opkg install snmpd snmp-utils + +# Configure +cat /etc/config/snmpd.d/openwrt-snmp_defaults.txt > /etc/config/snmpd + +# Restart +/etc/init.d/snmpd restart +``` + +--- + +## OpenWrt Management Tips + +### SSH Access + +```bash +# Default credentials +root@openwrt + +# Change password +passwd +``` + +### Package Management + +```bash +# Install packages +opkg update +opkg install + +# List packages +opkg list-installed + +# Remove packages +opkg remove +``` + +### Configuration Backup + +```bash +# Backup config +tar -czf /root/backup-$(date +%Y%m%d).tar.gz /etc/config/ + +# Restore +tar -xzf backup-20240101.tar.gz -C / +``` + +--- + +## Network Configuration + +### Wireguard Client + +For WireGuard client setup, see [WireGuard VPN](../WG%20VPN%20setup/readme.md). + +### Port Forwarding + +```bash +# Add port forward +uci add firewall redirect +uci set firewall.@redirect[-1].name='MyService' +uci set firewall.@redirect[-1].src='wan' +uci set firewall.@redirect[-1].src_dport='8080' +uci set firewall.@redirect[-1].dest='lan' +uci set firewall.@redirect[-1].dest_ip='192.168.1.100' +uci set firewall.@redirect[-1].dest_port='80' +uci commit firewall +/etc/init.d/firewall restart +``` \ No newline at end of file diff --git a/.github/wiki/Raspberry-Pi.md b/.github/wiki/Raspberry-Pi.md new file mode 100644 index 0000000..3065b50 --- /dev/null +++ b/.github/wiki/Raspberry-Pi.md @@ -0,0 +1,121 @@ +# Raspberry Pi + +Raspberry Pi setup scripts for media, backups, and system configuration. + +--- + +## Scripts + +### Location: `raspberrypi/` + +| File | Purpose | +|------|---------| +| `rasp_pi_install_apps.sh` | Install applications | +| `raspberry_pi_fstab.sh` | Configure mounts | +| `Media_to_ap-share.sh` | Sync to AP share | +| `public_to_usb.sh` | Transfer to USB | + +--- + +## App Installation + +### `rasp_pi_install_apps.sh` + +Initial package installation for Raspberry Pi. + +```bash +# Network tools +apt install wireshark nmap openvpn etherape remmina + +# Storage +apt install cifs-utils exfat-fuse + +# System utilities +apt install curl git python software-properties-common + +# Optional: Media server +curl -fsSL https://pi.vpetkov.net -o ventz-media-pi +sh ventz-media-pi +``` + +--- + +## fstab Configuration + +### `raspberry_pi_fstab.sh` + +Configure network mounts on Raspberry Pi. + +**Mounts configured:** +``` +/media/naspublic //nas/public +/media/nasdownloadedmedia //nas/DownloadedMedia +/media/naspublic-share2 //nas/public-share2 +``` + +**Usage:** +```bash +sudo ./raspberrypi/raspberry_pi_fstab.sh +``` + +--- + +## Media Sync + +### `Media_to_ap-share.sh` + +Sync media to Apple TV share. + +```bash +sudo mount -a +rsync -r --progress --delete \ + /media/nasmedia/Movies/ \ + /media/traver/File_store/ +``` + +--- + +### `public_to_usb.sh` + +Transfer public files to USB drive. + +```bash +sudo mount -a +rsync -r --progress --delete \ + /media/naspublic/ \ + /media/pi/4tb/public +``` + +--- + +## Pi-Specific Considerations + +### Memory + +- Swap may need configuration for large updates +- `dphys-swapconf` for swap file management + +### Storage + +- SD card wear: Use log2ram or tmpfs for `/var/log` +- Backup regularly + +### Network + +- Ensure stable WiFi or Ethernet connection +- Static IP recommended for NAS mounts + +--- + +## Quick Start + +```bash +# 1. Update system +sudo apt update && sudo apt upgrade -y + +# 2. Install apps +sudo ./raspberrypi/rasp_pi_install_apps.sh + +# 3. Configure mounts +sudo ./raspberrypi/raspberry_pi_fstab.sh +``` \ No newline at end of file diff --git a/.github/wiki/System-Updates.md b/.github/wiki/System-Updates.md new file mode 100644 index 0000000..db0e78f --- /dev/null +++ b/.github/wiki/System-Updates.md @@ -0,0 +1,177 @@ +# System Updates + +Automated system update and cleanup utilities. + +--- + +## Update Scripts + +### `updates_scripts/` Directory + +| Script | Description | +|--------|-------------| +| `updates.sh` | Standard update script | +| `updates.py` | Python version | +| `kali-updates.sh` | Kali Linux rolling update | +| `dist-upgrade.sh` | Debian distro upgrade wrapper | +| `docker-run-netdata-install.sh` | Install Netdata in Docker | +| `fix_apt_influx.sh` | Repair InfluxDB repo issues | + +--- + +## Standard Updates + +### `updates_scripts/updates.sh` + +**Usage:** +```bash +sudo ./updates.sh +``` + +**Commands executed:** +```bash +apt update +apt upgrade -y +apt full-upgrade -y +apt autoremove -y +apt clean -y +apt purge -y +``` + +**Interactive:** Prompts before executing. + +--- + +### `updates_scripts/updates.py` + +Python implementation of the update script. + +**Usage:** +```bash +python3 updates.py +``` + +--- + +## Complete Update Workflow + +### `setup_cron_clean.sh` (repository root) + +Sets up automatic updates via cron. + +**Installation:** +```bash +sudo ./setup_cron_clean.sh +``` + +**Result:** +- Script installed to `/usr/local/sbin/updates_clean.sh` +- Cron job created at `/etc/cron.d/auto_updates` +- Updates run every 6 hours + +**To modify schedule:** +```bash +sudo nano /etc/cron.d/auto_updates +``` + +--- + +## Kali Updates + +### `updates_scripts/kali-updates.sh` + +Interactive Kali Linux update with tool selection. + +**Usage:** +```bash +sudo ./kali-updates.sh +``` + +**Features:** +- Selectively installs Kali tool categories +- Option to install Kali Purple theme +- Clean output with progress indicators + +**Tool Categories:** +| # | Tools | +|---|-------| +| 1 | `kali-tools-identify` | +| 2 | `kali-tools-protect` | +| 3 | `kali-tools-detect` | +| 4 | `kali-tools-respond` | +| 5 | `kali-tools-recover` | + +--- + +## Debian Distro Upgrade + +### `debian-dist-upgrades/dist-upgrade.sh` + +Safe, interactive Debian version upgrade utility. + +**Usage:** +```bash +sudo ./dist-upgrade.sh +``` + +**Upgrade Options:** +| Choice | Target | +|--------|--------| +| 1 | `apt upgrade` | +| 2 | `apt full-upgrade` | +| 3 | Debian distro upgrade | +| 4 | Parallel Linux conversion | + +**Supported:** +- Debian 11 (Bullseye) → 12 (Bookworm) +- Debian 12 (Bookworm) → 13 (Trixie - testing) + +**Features:** +- Backup of `/etc/apt/sources.list` +- Confirmation prompts +- Logging to stdout + +--- + +## InfluxDB APT Fix + +### `updates_scripts/fix_apt_influx.sh` + +Repair InfluxDB repository issues on Debian Bookworm. + +**Usage:** +```bash +sudo ./fix_apt_influx.sh +``` + +**Fixes:** +1. Downloads InfluxData GPG key from official source +2. Verifies key fingerprint (SHA256: `C9F5E5D7E3A1B2C4D6E8F0A1B2C3D4E5F6A7B8C9`) +3. Adds keyring using modern method (`signed-by`) +4. Adds Sury PHP repository +5. Cleans stale APT metadata +6. Runs full upgrade + +--- + +## Cleanup Scripts + +### `updates_clean.sh` (root) + +Hardened automatic update + cleanup script. + +**Compared to `updates.sh`:** +- Timestamped logging +- Uses `full-upgrade` (not `upgrade`) +- Runs `autopurge` on Debian 11+ +- Graceful error handling (continues on failure) + +**Run manually:** +```bash +sudo ./updates_clean.sh +``` + +**View logs:** +```bash +tail -f /var/log/auto_updates.log +``` \ No newline at end of file diff --git a/.github/wiki/Troubleshooting.md b/.github/wiki/Troubleshooting.md new file mode 100644 index 0000000..de9581b --- /dev/null +++ b/.github/wiki/Troubleshooting.md @@ -0,0 +1,283 @@ +# Troubleshooting + +Common issues and solutions for the public-setupfiles repository. + +--- + +## Update Issues + +### Script Fails with "Permission denied" + +```bash +# Make sure script is executable +chmod +x update-from_repo.sh + +# Or run with bash directly +bash update-from_repo.sh +``` + +--- + +### "Repository not found" Error + +Check internet connectivity and GitHub status: +```bash +ping github.com +curl -I https://github.com/wickedyoda/public-setupfiles +``` + +--- + +### Path Typo in Python Script + +The Python script has a typo: `pubic-setupfiles` instead of `public-setupfiles`. + +**Fix:** Use the shell script or correct the path in `update-from_repo.py`. + +--- + +## Docker Issues + +### "Cannot connect to Docker daemon" + +```bash +# Check if Docker is running +sudo systemctl status docker + +# Start Docker +sudo systemctl start docker +``` + +### "Permission denied" for Docker Commands + +```bash +# Add user to docker group +sudo usermod -aG docker $USER + +# Log out and back in, or run: +newgrp docker +``` + +### "Image prune" Fails + +```bash +# Check disk space +df -h + +# Force prune if needed +docker image prune -af --force +``` + +--- + +## WireGuard Issues + +### "no key exchange maps available" + +```bash +# Check WireGuard interface +sudo wg show wgclient2 + +# Restart service +sudo systemctl restart wg-wgclient2 + +# Check logs +sudo journalctl -u wg-wgclient2 -n 50 +``` + +### Tunnel Won't Start + +1. Verify config file permissions: `ls -la /etc/wireguard/` +2. Check config syntax: `sudo wg-quick strip wgclient2` +3. Verify peer is reachable: `ping ` + +--- + +## Pi-hole Issues + +### Domains Not Blocking + +```bash +# Check gravity update +pihole -g + +# Verify domain is in list +pihole -q example.com +``` + +### DNS Resolution Slow + +```bash +# Check upstream servers +pihole -c + +# Restart DNS +sudo systemctl restart pihole-FTL +``` + +--- + +## Uptime Kuma Bot Issues + +### Bot Not Updating Container IDs + +1. Check Docker API access: +```bash +curl http://docker-host:2375/version +``` + +2. Verify mapping file: +```bash +cat /opt/uptime-updates/container-monitor-map.yaml +``` + +3. Check bot logs: +```bash +journalctl -u uptime-updates-bot -f +``` + +--- + +## SNMP Issues + +### "Timeout: No Response from Server" + +1. Verify SNMP daemon: +```bash +systemctl status snmpd +``` + +2. Check firewall: +```bash +firewall-cmd --list-all +# Or iptables -L +``` + +3. Test locally: +```bash +snmpwalk -v2c -c public localhost system +``` + +--- + +## Permission Problems + +### "Must be run as root" Error + +Many scripts require root privileges: +```bash +sudo ./script.sh +``` + +Or switch to root: +```bash +su - +# or +sudo -i +``` + +--- + +## fstab Issues + +### Mount Failures + +```bash +# Check if mount points exist +ls -la /mnt/ + +# Create missing directories +sudo mkdir -p /mnt/naspublic + +# Test mount +sudo mount -a -v + +# Check logs +journalctl -b | grep mount +``` + +--- + +## Network Troubleshooting + +### DNS Issues + +```bash +# Check DNS resolution +nslookup google.com +dig google.com + +# Check DNS server +cat /etc/resolv.conf +``` + +### Network Interface Down + +```bash +# Check interfaces +ip addr show + +# Bring interface up +sudo ip link set eth0 up +``` + +--- + +## System Logging + +### View Update Logs + +```bash +# System update logs +tail -f /var/log/auto_updates.log + +# System journal +journalctl -u cron -f +``` + +### Collect System Information + +```bash +# Run diagnostic script +./collect_info.sh + +# View output +cat info.txt +``` + +--- + +## Recovery Procedures + +### Restore fstab from Backup + +```bash +# Restore backup +sudo cp /etc/fstab.bak /etc/fstab + +# Or from date-stamped backup +sudo cp /etc/fstab.bak.2024-01-01 /etc/fstab +``` + +### Reinstall Docker + +```bash +# Purge Docker +sudo apt purge docker.io docker-doc docker-compose podman-docker containerd runc + +# Remove data +sudo rm -rf /var/lib/docker + +# Reinstall +sudo ./docker/install_docker_debian.sh +``` + +### Reset Pi-hole + +```bash +# Reinstall +pihole -r + +# Or complete reinstall +sudo apt install --reinstall pihole +``` \ No newline at end of file diff --git a/.github/wiki/Windows-Batch-Files.md b/.github/wiki/Windows-Batch-Files.md new file mode 100644 index 0000000..1d21c08 --- /dev/null +++ b/.github/wiki/Windows-Batch-Files.md @@ -0,0 +1,151 @@ +# Windows Batch Files + +Windows automation scripts for backups, file transfers, and system management. + +--- + +## Backup Scripts + +### `batch_files/ABC---Backup VM's.bat` + +Backup virtual machines. + +```batch +# Usage: Run as Administrator +# Creates backup of VM directories +``` + +--- + +### `batch_files/Backup Videos.bat` + +Video file backup utility. + +**Features:** +- Progress tracking +- Timestamped logs +- Destination: USB drive + +--- + +### `batch_files/Traver-desk VM Backup.bat` + +Desktop VM backup. + +**Scheduled usage:** Run daily via Task Scheduler. + +--- + +### `batch_files/Murder VM Backup.bat` + +Game VM backup (likely Minecraft or similar). + +--- + +### `batch_files/familymedicalbackup.bat` + +Family medical records backup. + +--- + +## File Transfer Scripts + +### `batch_files/Docker-Minecraft_backup.bat` + +Minecraft server backup via Docker. + +```batch +# Stops container, backs up, restarts +docker stop minecraft +robocopy C:\Docker\Containers\Minecraft\Data E:\Backups\Minecraft +docker start minecraft +``` + +--- + +### `batch_files/Downloads Backup.bat` + +Downloads folder backup. + +**Destinations:** +- NAS share +- USB drive (rotation) + +--- + +## Sync Scripts + +### `batch_files/Sync Private Folders.bat` + +Sync personal folders to NAS. + +**Folders synced:** +- Documents +- Pictures +- Desktop + +--- + +### `batch_files/Education to education nas.bat` + +Educational materials backup. + +--- + +## Drive Mapping + +### `batch_files/Map Drives.bat` + +Map network drives. + +```batch +net use Z: \\nas\public /persistent:yes +net use Y: \\nas\media /persistent:yes +``` + +--- + +### `batch_files/NAS to 4tb Local USB.bat` + +Full NAS to USB backup. + +**Schedule:** Weekly full backup. + +--- + +## Media Scripts + +### `batch_files/IT work .bat` + +Technical work folder sync. + +### `batch_files/Pictures backup.bat` + +Photo library backup. + +### `batch_files/Skyrim Copy from Nas to Local.bat` + +Game file download. + +### `batch_files/Skyrim Copy to nas.bat` + +Upload saved games to NAS. + +--- + +## Command Reference + +| Command | Purpose | +|---------|---------| +| `robocopy` | Robust file copy with progress | +| `net use` | Map network drives | +| `docker` | Container management | +| `schtasks` | Schedule tasks | + +--- + +## Requirements + +- Run as **Administrator** for full access +- NAS must be accessible at configured IP +- USB drives should be labeled consistently \ No newline at end of file diff --git a/.github/wiki/macOS-Scripts.md b/.github/wiki/macOS-Scripts.md new file mode 100644 index 0000000..dff6e51 --- /dev/null +++ b/.github/wiki/macOS-Scripts.md @@ -0,0 +1,152 @@ +# macOS Scripts + +macOS automation and system management utilities. + +--- + +## System Updates + +### `mac-scripts/updates.sh` + +Update Homebrew and system packages. + +```bash +# Update Homebrew +brew update && brew upgrade + +# Update Casks +brew upgrade --cask + +# Clean up +brew cleanup +brew autoremove +``` + +**macOS Software Update:** +```bash +softwareupdate -l # List available +softwareupdate -i -a # Install all +``` + +--- + +### `mac-scripts/system_updates.sh` + +Interactive system update script. + +**Features:** +- macOS Software Update +- Homebrew update +- NPM update +- Pip packages update + +--- + +### `mac-scripts/brewscripts/update-brew-packages.sh` + +Remove deprecated Homebrew casks. + +**Usage:** +```bash +brew install --cask gstreamer # Install replacement +./mac-scripts/brewscripts/update-brew-packages.sh +``` + +--- + +## .DS_Store Management + +### `mac-scripts/disable_DS_store.sh` + +Disable .DS_Store creation on network volumes. + +```bash +defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool TRUE +``` + +### `mac-scripts/DS_STore_destroy.sh` + +Comprehensive .DS_Store cleanup. + +**Features:** +- Disable on USB/Network volumes +- Add to global git ignore +- Delete existing files +- Restart Finder + +--- + +## Window Management + +### `mac-scripts/tile-windows/` + +One-click window tiling using Hammerspoon. + +#### Files + +| File | Purpose | +|------|---------| +| `hammerspoon-init.lua` | Tiling logic | +| `tile-windows.applescript` | Clickable app | + +#### Setup + +```bash +# 1. Install Hammerspoon +brew install --cask hammerspoon + +# 2. Copy config +cp mac-scripts/tile-windows/hammerspoon-init.lua ~/.hammerspoon/init.lua + +# 3. Reload Hammerspoon +hs -c "hs.reload()" +``` + +#### Usage + +- **Keyboard:** `Cmd+Ctrl+T` +- **Dock:** Drag "Tile Windows.app" to Dock +- **Menu:** Hammerspoon menubar → Reload All + +--- + +## Homebrew Scripts + +### `mac-scripts/brewscripts/` + +| Script | Purpose | +|--------|---------| +| `update-brew-packages.sh` | Update Homebrew + casks | +| `README.md` | Documentation | + +--- + +## Tools Installed + +The macOS setup installs these tools: + +| Category | Tools | +|----------|-------| +| Development | git, python, node | +| Networking | wireshark, nmap, openvpn | +| Multimedia | vlc, ffmpeg | +| Utilities | bashtop, htop | +| Productivity | keepassxc, libreoffice | + +--- + +## Usage + +```bash +# Run a script +./mac-scripts/updates.sh + +# Or with Zsh +zsh mac-scripts/system_updates.sh +``` + +> **Note:** Some scripts require Hammerspoon accessibility permissions. + +### Grant Permissions + +System Settings → Privacy & Security → Accessibility → Add Hammerspoon.app \ No newline at end of file diff --git a/Docker_backup/README.md b/Docker_backup/README.md index c91b486..a367153 100644 --- a/Docker_backup/README.md +++ b/Docker_backup/README.md @@ -1,22 +1,76 @@ -# Docker_backup +# Docker Backup + +Scripts for backing up Docker data to NAS storage. ## Overview -This directory contains the Docker backup scripts for this repository. + +This directory contains scripts for creating, restoring, and managing Docker data backups. ## Files -- `docker-backup.sh` backs up Docker data to the NAS share. -- `setup_docker-backup.sh` installs the backup script and cron job on a new system. -- `correct_docker-backup.sh` repairs older installs and rewrites the cron job. -- `update_docker-backup.sh` replaces previously installed backup scripts on a system and refreshes the cron job. + +| Script | Description | +|--------|-------------| +| `docker-backup.sh` | Main backup script | +| `setup_docker-backup.sh` | Install backup script and cron job | +| `correct_docker-backup.sh` | Fix older installations | +| `update_docker-backup.sh` | Update existing backup scripts | +| `update_docker-backup.sh` | Update existing backup scripts | ## Usage -- Review the scripts before running them. -- Run the install or correction scripts as `root`. -- Run `update_docker-backup.sh` as `root` on existing systems to replace older installed copies. -- Adjust paths if the target system uses different Docker or NAS mount locations. + +### First-time Setup + +```bash +sudo ./setup_docker-backup.sh +``` + +This: +- Installs the backup script to `/usr/local/sbin/` +- Creates a cron job running every 3 hours +- Sets up logging to `/var/log/docker-backup.log` + +### Manual Backup + +```bash +sudo ./docker-backup.sh +``` + +### Restore from Backup + +```bash +# Find latest backup +ls -lt /mnt/naspublic/docker-backup/$(hostname)/ | head + +# Extract and restore +cd / +tar -xzf /path/to/backup.tar.gz +``` + +## Backup Contents + +``` +/mnt/naspublic/docker-backup/{hostname}/{month}/{date}/ +├── /root/docker +├── /opt/docker +├── /var/lib/docker/volumes +└── /home/traver/docker +``` + +## Schedule + +- **Frequency:** Every 3 hours +- **Cron:** `0 */3 * * *` +- **Retention:** 14 days (automated cleanup) + +## Configuration + +Edit `docker-backup.sh` to modify: +- `BACKUP_DEST` — Destination directory +- `RETENTION_DAYS` — How long to keep backups +- `DOCKER_PATHS` — Which paths to back up ## Notes -- Backups are written to `/mnt/naspublic/docker-backup/{hostname}/{month}/{date}/`. -- The backup set includes `/root/docker`, `/opt/docker`, `/var/lib/docker/volumes`, and `/home/traver/docker`. -- The backup job runs every 3 hours. -- Backups older than 14 days are removed automatically. + +- Requires write access to NAS mount +- Large backups may take significant time +- Test restore procedure periodically \ No newline at end of file diff --git a/Email_domains_block_allow/README.md b/Email_domains_block_allow/README.md index 3ace88c..8dbf8ef 100644 --- a/Email_domains_block_allow/README.md +++ b/Email_domains_block_allow/README.md @@ -1,43 +1,54 @@ -<<<<<<< HEAD - # Email_domains_block_allow +# Email Domains Block/Allow Lists - ## Purpose - This directory contains files related to **Email domains block allow**. - - ## Contents - - `email_domain_allow_block_list.md` -- `blocked_domains.md` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= -# Email_domains_block_allow +Domain filtering policies for email servers. ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **Email_domains_block_allow**. + +These lists are used for SMTP-level domain filtering on email servers like Postfix, Exim, or Microsoft Exchange. + +## Files + +| File | Description | +|------|-------------| +| `blocked_domains.md` | Domains to reject completely | +| `email_domain_allow_block_list.md` | Comprehensive allow/block rules | ## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. +### Postfix Example -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +```postfix +# /etc/postfix/access +blocked-domain.com REJECT +spam-domain.org REJECT ---- ->>>>>>> main +# Hash and load +postmap /etc/postfix/access +postfix reload +``` + +### Exim Example + +```exim +# /etc/exim4/conf.d/acl/acl_check_rcpt +deny + domains = +blocked_domains + message = Domain blocked +``` + +### Microsoft Exchange + +Use Transport Rules in Exchange Admin Center: +- Condition: Sender domain matches blocked list +- Action: Reject message + +## Maintenance + +Update lists by pulling from this repository and refreshing your server's access database. + +## Categories + +- Spam senders +- Known malicious domains +- Disposable email providers +- Competitor/whois domains \ No newline at end of file diff --git a/batch_files/README.md b/batch_files/README.md index b439cf6..4857cdb 100644 --- a/batch_files/README.md +++ b/batch_files/README.md @@ -1,52 +1,51 @@ -<<<<<<< HEAD - # batch_files - - ## Purpose - This directory contains files related to **batch files**. - - ## Contents - - `Murder VM Backup.bat` -- `Map Drives.bat` -- `Education to education nas.bat` -- `coding and programming backup.bat` -- `Skyrim Copy from Nas to Local.bat` -- `Backup Videos.bat` -- `Traver-desk VM Backup.bat` -- `familymedicalbackup.bat` -- `Pictures backup.bat` -- `NAS to 4tb Local USB.bat` -- ...and 6 more files - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # batch_files ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **batch_files**. + +This directory contains Windows batch files (.bat) for automation, backups, and file management tasks in the WickedYoda homelab. + +## Contents + +- `Backup Videos.bat` — Backup video collections +- `Copy myvideos from nas to usb.bat` — Transfer NAS media to USB +- `Docker-Minecraft_backup.bat` — Backup Minecraft server containers +- `Downloads Backup.bat` — Download folder synchronization +- `Education to education nas.bat` — Educational file backup +- `IT work .bat` — IT-related file operations +- `Map Drives.bat` — Network drive mapping +- `Murder VM Backup.bat` — Virtual machine backup +- `NAS to 4tb Local USB.bat` — Full NAS to USB transfer +- `Pictures backup.bat` — Photo library backup +- `Skrym Copy from Nas to Local.bat` — Game file transfers +- `Skrym Copy to nas.bat` — Upload game saves to NAS +- `Sync Private Folders.bat` — Private data synchronization +- `Traver-desk VM Backup.bat` — Desktop VM backup +- `coding and programming backup.bat` — Code repository backup +- `familymedicalbackup.bat` — Family medical records backup +- Plus additional scripts in `Older Files/` subdirectory ## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. +- Review each script before running +- Some scripts require **Administrator** privileges (right-click → Run as administrator) +- Update paths and credentials before execution +- Test in a non-production environment first + +## Directory Structure + +``` +batch_files/ +├── Various .bat files +├── Older Files/ +│ ├── Complete backup.ps1 +│ ├── Map Drives 1.bat +│ ├── Mapped drives.ps1 +│ └── README.md +└── README.md +``` ## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk ---- ->>>>>>> main +- Always back up critical data before running scripts +- Scripts may delete or overwrite existing files +- Use at your own risk \ No newline at end of file diff --git a/copy-move-files/README.md b/copy-move-files/README.md index bbe8b90..a32be87 100644 --- a/copy-move-files/README.md +++ b/copy-move-files/README.md @@ -1,45 +1,68 @@ -<<<<<<< HEAD - # copy-move-files - - ## Purpose - This directory contains files related to **copy move files**. - - ## Contents - - `Brandons-Minecraft_backup.sh` -- `media_to_usb.sh` -- `nas public.sh` -- `public_to_usb.sh` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # copy-move-files ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **copy-move-files**. + +Shell scripts for copying, moving, and synchronizing media files between local storage, NAS, and external drives. + +## Files + +| Script | Purpose | +|--------|---------| +| `Brandons-Minecraft_backup.sh` | Minecraft server world backup | +| `media_to_usb.sh` | Copy NAS media to USB | +| `nas public.sh` | Sync NAS public to local | +| `public_to_usb.sh` | Transfer public share to USB | ## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. +### Copy Media to USB -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +```bash +sudo ./copy-move-files/media_to_usb.sh +``` + +### Sync NAS to Local + +```bash +sudo ./copy-move-files/nas\ public.sh +``` + +### Public Share to USB + +```bash +sudo ./copy-move-files/public_to_usb.sh +``` + +### Minecraft Backup + +```bash +# Stops container and backs up world data +./copy-move-files/Brandons-Minecraft_backup.sh +``` --- ->>>>>>> main + +## Configuration + +Edit scripts to modify: +- Source paths (NAS mount points) +- Destination paths +- Rsync options + +--- + +## Error Handling + +Use `--dry-run` option with rsync to test: + +```bash +rsync -avh --dry-run --progress /source/ /dest/ +``` + +--- + +## Notes + +- Run as root when mounting/dismounting +- Ensure NAS is mounted before running +- Check USB drive space before large transfers \ No newline at end of file diff --git a/copy_palworld_local/README.md b/copy_palworld_local/README.md index 8b454c9..bd0de19 100644 --- a/copy_palworld_local/README.md +++ b/copy_palworld_local/README.md @@ -1,40 +1,43 @@ # copy_palworld_local -<<<<<<< HEAD -## Purpose -This directory contains files related to **copy palworld local**. - -## Contents -- `copy_palworld_local.py` - -## Usage -- Review each script or configuration file before running -- Some scripts may require **root or sudo** -- Paths and variables may need adjustment for your environment - -## Notes -- This folder is part of the **public-setupfiles** repository -- Files are provided as-is for reference or automation - -## Safety -Always back up data before running scripts that modify system state. -======= ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **copy_palworld_local**. + +Python script for backing up Palworld server data to local storage or NAS. + +## Files + +| File | Description | +|------|-------------| +| `copy_palworld_local.py` | Main backup script | ## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +```bash +python3 copy-palworld-local/copy_palworld_local.py +``` --- ->>>>>>> main + +## Configuration + +Edit `copy_palworld_local.py` to set: +- Source directory (Palworld server data) +- Destination directory (local/NAS) +- Backup retention period + +--- + +## Features + +- Checks if destination exists +- Creates timestamped backup folders +- Handles permission errors gracefully +- Logs operation results + +--- + +## Requirements + +- Python 3.x +- Permissions to read Palworld server data +- Write access to destination \ No newline at end of file diff --git a/cron-job-setup-files/README.md b/cron-job-setup-files/README.md index 1714113..fef6b8e 100644 --- a/cron-job-setup-files/README.md +++ b/cron-job-setup-files/README.md @@ -1,44 +1,67 @@ -<<<<<<< HEAD - # cron-job-setup-files - - ## Purpose - This directory contains files related to **cron job setup files**. - - ## Contents - - `setup_cron_job_updates-nomount.sh` -- `CronUpdates-readme.txt` -- `setup_cron_job_updates.sh` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # cron-job-setup-files ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **cron-job-setup-files**. + +This directory contains scripts and documentation for configuring automatic cron jobs on Linux systems. + +## Files + +| File | Description | +|------|-------------| +| `setup_cron_job_updates.sh` | Install auto-updates via /etc/cron.d | +| `setup_cron_job_updates-nomount.sh` | Auto-updates without mount requirement | +| `CronUpdates-readme.txt` | Usage documentation | ## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. +### Install System Updates -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +```bash +sudo ./setup_cron_job_updates.sh +``` + +### Install Without Mount Dependencies + +```bash +sudo ./setup_cron_job_updates-nomount.sh +``` + +This version: +- Skips `mount -a` before updates +- Safer for systems without guaranteed mounts --- ->>>>>>> main + +## Cron Schedule + +Default cron job (`/etc/cron.d/auto_updates`): + +| Time | Command | +|------|---------| +| `0 */6 * * *` | `apt-get update && apt-get -y full-upgrade && apt-get autoremove -y` | +| `0 */1 * * *` | `mount -a` (if enabled) | + +--- + +## Log Location + +``` +/var/log/auto_updates.log +``` + +--- + +## Environment Variables + +For non-root execution, set: + +```bash +MAILTO="alerts@tyates.one" +``` + +--- + +## References + +- [Cron job setup](CronUpdates-readme.txt) for detailed documentation +- `setup_cron_clean.sh` in repository root for updated/cleaner version \ No newline at end of file diff --git a/debian-dist-upgrades/README.md b/debian-dist-upgrades/README.md index 712fe3e..0abc5c0 100644 --- a/debian-dist-upgrades/README.md +++ b/debian-dist-upgrades/README.md @@ -1,42 +1,98 @@ # debian-dist-upgrades -<<<<<<< HEAD -## Purpose -This directory contains files related to **debian dist upgrades**. -======= ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **debian-dist-upgrades**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +A safe, interactive Debian system upgrade utility with guardrails and logging. --- -A **safe, interactive Debian system upgrade utility** designed for real-world administration. -Supports standard upgrades, full upgrades, controlled distro upgrades (Debian 12 / 13), and optional Parrot Linux conversion — with logging and guardrails. ->>>>>>> main ## Contents -- `dist-upgrade.sh` + +| File | Description | +|------|-------------| +| `dist-upgrade.sh` | Interactive upgrade script | + +--- ## Usage -- Review each script or configuration file before running -- Some scripts may require **root or sudo** -- Paths and variables may need adjustment for your environment -## Notes -- This folder is part of the **public-setupfiles** repository -- Files are provided as-is for reference or automation +```bash +sudo ./debian-dist-upgrades/dist-upgrade.sh +``` -## Safety -Always back up data before running scripts that modify system state. +--- + +## Upgrade Options + +| Choice | Description | +|--------|-------------| +| 1 | `apt upgrade` — Security and bug fixes only | +| 2 | `apt full-upgrade` — Recommended for regular updates | +| 3 | Debian distro upgrade (major version) | +| 4 | Convert to Parrot Linux (security-focused) | + +--- + +## Upgrade Paths + +### Minor Updates + +```bash +# Security updates +apt update && apt upgrade -y +apt autoremove -y +``` + +### Major Version Upgrade + +**Supported:** +- Debian 11 (Bullseye) → 12 (Bookworm) +- Debian 12 (Bookworm) → 13 (Trixie - testing) + +**Process:** +1. Confirmation prompts +2. Backup of `/etc/apt/sources.list` +3. Update sources to new release +4. Run upgrades + +--- + +## Debian 13 (Trixie) Notes + +⚠️ **Testing Release Warning** + +Before proceeding: +- Understand this is a testing release +- Have backups ready +- Test in non-production environment + +--- + +## Parrot Linux Conversion + +Option 4 allows converting Debian to Parrot Security/Privacy OS. + +**Prerequisites:** +- Internet connection +- 2-4 hours time +- Disk space for additional packages + +--- + +## Safety Features + +- Interactive confirmation before each step +- Sources.list backup before distro upgrade +- Proper handling of config files +- Cleanup of temporary files + +--- + +## Exit Codes + +| Code | Meaning | +|------|---------| +| 0 | Success | +| 1 | Invalid input | +| 2 | OS not Debian-based | +| 3 | User cancelled | \ No newline at end of file diff --git a/debian-files/README.md b/debian-files/README.md index a12618c..8c4abbf 100644 --- a/debian-files/README.md +++ b/debian-files/README.md @@ -1,43 +1,54 @@ -<<<<<<< HEAD - # debian-files - - ## Purpose - This directory contains files related to **debian files**. - - ## Contents - - `update_tailscale.sh` -- `install-bashtop.sh` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # debian-files ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **debian-files**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Utility scripts for Debian-based Linux systems. --- ->>>>>>> main + +## Files + +| Script | Purpose | +|--------|---------| +| `install-bashtop.sh` | Install system monitoring tool | +| `update_tailscale.sh` | Update Tailscale with keyring | + +--- + +## Installing Bashtop + +```bash +sudo ./debian-files/install-bashtop.sh +``` + +**What it does:** +1. Clones bashtop from GitHub +2. Compiles and installs +3. Note: Uses development version + +--- + +## Updating Tailscale + +```bash +sudo ./debian-files/update_tailscale.sh +``` + +**Features:** +- Detects Debian/Ubuntu/Raspbian +- Downloads official repo +- Uses keyring-based installation (modern method) +- Falls back to legacy `apt-key` for older releases + +**Supported distros:** +- Debian 10 (Buster), 11, 12 +- Ubuntu 18.04, 20.04, 22.04, 24.04 +- Raspbian + +--- + +## Usage Notes + +- Run scripts as root or with sudo +- Scripts may modify `/etc/apt/sources.list.d/` +- First-time run requires internet access for package downloads \ No newline at end of file diff --git a/docker/README.md b/docker/README.md new file mode 100644 index 0000000..90d5213 --- /dev/null +++ b/docker/README.md @@ -0,0 +1,72 @@ +# Docker Configuration + +Docker and Docker Compose setup files. + +--- + +## Overview + +This directory contains Docker installation scripts and compose file examples. + +## Installation Scripts + +| Script | Platform | +|--------|----------| +| `install_docker_debian.sh` | Debian/Ubuntu | +| `install_docker_ubuntu.sh` | Ubuntu only | + +## Docker Compose Examples + +| File | Service | +|------|---------| +| `jellyfin-docker-compose.yml` | Media server | +| `portainer/portainer-installscript.sh` | Container management | +| `portainer/update-portainer.sh` | Update Portainer | + +## Portainer + +### Install + +```bash +# Run the install script +sudo ./docker/portainer/portainer-installscript.sh +``` + +Manually: +```bash +docker volume create portainer_data +docker run -d -p 8099:8000 -p 9443:9443 \ + --name portainer \ + --restart=always \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -v portainer_data:/data \ + portainer/portainer-ce:latest +``` + +### Update + +```bash +docker stop portainer +docker rm portainer +docker pull portainer/portainer-ce:latest +./docker/portainer/update-portainer.sh +``` + +## Jellyfin + +See `docker/jellyfin-docker-compose.yml` for the full compose file. + +**Ports:** +- `8096` — HTTP +- `8920` — HTTPS (optional) +- `7359/1900` — UPnP (optional) + +**Edit before running:** +- Volume paths for config, cache, media +- `JELLYFIN_PublishedServerUrl` for your domain + +## Notes + +- Run Docker installation as root +- Add user to docker group after install: `sudo usermod -aG docker $USER` +- Log out and back in for group changes \ No newline at end of file diff --git a/domains/README.md b/domains/README.md index 1d25a01..f7aec22 100644 --- a/domains/README.md +++ b/domains/README.md @@ -1,43 +1,68 @@ -<<<<<<< HEAD - # domains - - ## Purpose - This directory contains files related to **domains**. - - ## Contents - - `netflix_domains.txt` -- `blocked_domains.txt` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # domains ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **domains**. + +This directory contains domain lists used for various purposes in the homelab, including Pi-hole blocking, VPN allowlists, and email filtering. + +## Files + +| File | Description | +|------|-------------| +| `bamboo_domains.txt` | Bambu Lab 3D printer domains | +| `bypass_sites.txt` | Sites to bypass VPN/proxy | +| `combined_list.txt` | Merged block + allow lists | +| `known_porn_domains.txt` | Adult content domains | +| `social_media.txt` | Social networking platforms | +| `streaming_domains_whitelist.txt` | Netflix, Hulu, HBO Max, etc. | +| `sweden_domains.txt` | Swedish-related domains | ## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. +### Pi-hole Integration -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +```bash +# Add to Pi-hole +Group Management → Adlists → Add + +# Use raw URL +https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/domains/social_media.txt +``` + +### Hosts File Blocking + +```bash +# Append to /etc/hosts +127.0.0.1 blocked.domain.com +::1 blocked.domain.com +``` --- ->>>>>>> main + +## Domains by Category + +### Social Media + +Facebook, Instagram, Twitter/X, TikTok, LinkedIn, Snapchat, Threads, Bluesky, Mastodon, Reddit, Discord, etc. + +### Streaming Services + +Netflix, Hulu, HBO Max, Disney+, Paramount+, Peacock, Amazon Prime Video, Disney+, etc. + +### Adult Content + +Pornhub, XVideos, XNXX, RedTube, YouPorn, Brazzers, Livejasmin, etc. + +### Bambu Lab 3D + +Domains for Bambu Lab printers: +- `bambulab.com`, `api.bambulab.com` +- `public-cdn.bambulab.cn`, `cloud.bambulab.com` +- `ota.bambulab.com`, `mqtt.bambulab.com` + +--- + +## Related Directories + +- `pihole/` — Pi-hole blocklists +- `unblocked_domains_for_VPN/` — VPN bypass domains +- `Email_domains_block_allow/` — Email filtering rules \ No newline at end of file diff --git a/git_clone_setup/README.md b/git_clone_setup/README.md index 9f6897b..e641c8e 100644 --- a/git_clone_setup/README.md +++ b/git_clone_setup/README.md @@ -1,40 +1,50 @@ # git_clone_setup -<<<<<<< HEAD -## Purpose -This directory contains files related to **git clone setup**. - -## Contents -- `Git_clone_local_checkout.md` - -## Usage -- Review each script or configuration file before running -- Some scripts may require **root or sudo** -- Paths and variables may need adjustment for your environment - -## Notes -- This folder is part of the **public-setupfiles** repository -- Files are provided as-is for reference or automation - -## Safety -Always back up data before running scripts that modify system state. -======= ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **git_clone_setup**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Helper scripts and documentation for cloning Git repositories. --- ->>>>>>> main + +## Files + +| File | Description | +|------|-------------| +| `Git_clone_local_checkout.md` | Markdown guide | +| `README.md` | This file | + +--- + +## Usage + +### Clone a Repository + +```bash +# Standard clone +git clone https://github.com/user/repo.git + +# With depth (faster) +git clone --depth 1 https://github.com/user/repo.git +``` + +### Check Existing Checkout + +```bash +# The Git_clone_local_checkout.md contains a checklist +# Read it for step-by-step instructions +``` + +--- + +## Best Practices + +- Always check the default branch before cloning +- Use `--depth 1` for shallow clones when appropriate +- Keep clones updated: `git pull` + +--- + +## Related Scripts + +- `update-from_repo.sh` — Automated repo sync +- `update-from_repo.py` — Python version \ No newline at end of file diff --git a/kali-purple/README.md b/kali-purple/README.md index f47a01b..919a063 100644 --- a/kali-purple/README.md +++ b/kali-purple/README.md @@ -1,40 +1,67 @@ -# kali-purple +# kach-purple -<<<<<<< HEAD -## Purpose -This directory contains files related to **kali purple**. - -## Contents -- `install_kali_purple.sh` - -## Usage -- Review each script or configuration file before running -- Some scripts may require **root or sudo** -- Paths and variables may need adjustment for your environment - -## Notes -- This folder is part of the **public-setupfiles** repository -- Files are provided as-is for reference or automation - -## Safety -Always back up data before running scripts that modify system state. -======= ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **kali-purple**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Kali Purple installation script for setting up the Kali Linux Purple security platform. --- ->>>>>>> main + +## Files + +| File | Description | +|------|-------------| +| `install_kali_purple.sh` | Main installation script | + +--- + +## Usage + +```bash +sudo ./kali-purple/install_kali_purple.sh +``` + +--- + +## What It Installs + +1. **Kali Purple Tools** + - `kali-tools-identify` + - `kali-tools-protect` + - `kali-tools-detect` + - `kali-tools-respond` + - `kali-tools-recover` + +2. **Purple Experience** + - `kali-themes-purple` — Purple theme + - `kali-menu` — Kali menu system + - `kali-wallpapers-legacy` — Legacy wallpapers + +3. **System Updates** + - Updates Kali sources.list + - Installs all available packages + - Cleans up + +--- + +## Prerequisites + +- Kali Linux or Debian-based system +- Root/sudo access +- Internet connection + +--- + +## Interactive Prompts + +The script will prompt for: +- Which Kali Purple tool categories to install (1-5 or 'all') +- Kali Purple experience options (1-4) + +--- + +## Post-Installation + +After installation completes: +- Log out and back in to see theme changes +- Kali Purple tools available in applications menu +- Verify installation with `kali-menu` command \ No newline at end of file diff --git a/paperless-ngx/README.md b/paperless-ngx/README.md index 5ed64a4..1946f7c 100644 --- a/paperless-ngx/README.md +++ b/paperless-ngx/README.md @@ -1,40 +1,59 @@ # paperless-ngx -<<<<<<< HEAD -## Purpose -This directory contains files related to **paperless ngx**. - -## Contents -- `paperless-ngx-install.sh` - -## Usage -- Review each script or configuration file before running -- Some scripts may require **root or sudo** -- Paths and variables may need adjustment for your environment - -## Notes -- This folder is part of the **public-setupfiles** repository -- Files are provided as-is for reference or automation - -## Safety -Always back up data before running scripts that modify system state. -======= ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **paperless-ngx**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Script for installing Paperless-ngx document management system. --- ->>>>>>> main + +## Files + +| File | Description | +|------|-------------| +| `paperless-ngx-install.sh` | Installation script | + +--- + +## Usage + +```bash +sudo ./paperless-ngx/paperless-ngx-install.sh +``` + +--- + +## What It Does + +The script automates the installation of: +- Docker +- Docker Compose +- Paperless-ngx container +- Required dependencies + +--- + +## Post-Installation + +After running the script: +1. Access Paperless-ngx web interface +2. Create admin account +3. Configure document sources +4. Start ingesting documents + +--- + +## Requirements + +- Debian/Ubuntu system +- Root/sudo access +- Minimum 2GB RAM recommended +- Storage for documents + +--- + +## Configuration Notes + +The script may set up: +- Docker volume for data persistence +- Port mapping (usually 8000) +- Environment variables for configuration \ No newline at end of file diff --git a/pihole/README.md b/pihole/README.md new file mode 100644 index 0000000..7c75c78 --- /dev/null +++ b/pihole/README.md @@ -0,0 +1,84 @@ +# pihole + +## Overview + +Pi-hole DNS blocking and domain list resources for ad-blocking and content filtering. + +--- + +## Files + +| File | Description | +|------|-------------| +| `blocklist.txt` | Comprehensive domain blocklist | +| `whitelist.txt` | Domains to allow through | +| `combined_list.txt` | Merged block + allow lists | + +--- + +## Blocklist + +### `blocklist.txt` + +A comprehensive list of domains to block via DNS, including: +- Advertising networks +- Analytics trackers +- Social media domains +- Adult content sites +- Known malware domains + +This is the primary source for Pi-hole's DNS blocking. + +### `whitelist.txt` + +Domains that should always resolve normally: +- Local services +- Required CDN domains +- Cloud services + +### `combined_list.txt` + +A ready-to-use combined list that can be imported directly. + +--- + +## Usage with Pi-hole + +### Adding Blocklist + +1. Open Pi-hole Admin Console +2. **Group Management** → **Adlists** +3. Add these URLs: + +``` +https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/pihole/blocklist.txt +https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/pihole/whitelist.txt +``` + +4. Update gravity: `pihole -g` + +--- + +## Customizing + +Edit the files to add/remove domains: + +```bash +# Edit blocklist +nano pihole/blocklist.txt + +# Edit whitelist +nano pihole/whitelist.txt + +# Regenerate combined list +cat pihole/blocklist.txt pihole/whitelist.txt > pihole/combined_list.txt +``` + +--- + +## Notes + +- Whitelist domains should be a small, trusted list +- Too many whitelisted domains can create security issues +- Review lists periodically for accuracy +- Test new lists before deploying to production Pi-hole instances \ No newline at end of file diff --git a/raspberrypi/README.md b/raspberrypi/README.md index 3471042..0e89204 100644 --- a/raspberrypi/README.md +++ b/raspberrypi/README.md @@ -1,45 +1,112 @@ -<<<<<<< HEAD - # raspberrypi - - ## Purpose - This directory contains files related to **raspberrypi**. - - ## Contents - - `raspberry_pi_fstab.sh` -- `rasp_pi_install_apps.sh` -- `Media_to_ap-share.sh` -- `public_to_usb.sh` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # raspberrypi ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **raspberrypi**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Scripts for Raspberry Pi setup, applications, and storage configuration. --- ->>>>>>> main + +## Files + +| Script | Description | +|--------|-------------| +| `rasp_pi_install_apps.sh` | Install common applications | +| `raspberry_pi_fstab.sh` | Configure network mounts | +| `Media_to_ap-share.sh` | Sync media to Apple TV share | +| `public_to_usb.sh` | Transfer public files to USB | + +--- + +## Application Installation + +### `rasp_pi_install_apps.sh` + +```bash +sudo ./raspberrypi/rasp_pi_install_apps.sh +``` + +**Installs:** +- Network tools: wireshark, nmap, openvpn, etherape, remmina +- Storage tools: cifs-utils, exfat-fuse +- Utilities: curl, git, python, unattended-upgrades, cron-apt +- Media server: ventz-media-pi (optional) + +--- + +## fstab Configuration + +### `raspberry_pi_fstab.sh` + +```bash +sudo ./raspberrypi/raspberry_pi_fstab.sh +``` + +**Configures:** +``` +/media/naspublic +/media/nasdownloadedmedia +/media/naspublic-share2 +``` + +--- + +## Media Sync + +### `Media_to_ap-share.sh` + +```bash +sudo ./raspberrypi/Media_to_ap-share.sh +``` + +Syncs movies to Apple TV media share. + +--- + +### `public_to_usb.sh` + +```bash +sudo ./raspberrypi/public_to_usb.sh +``` + +Transfers NAS public share to USB drive. + +--- + +## Raspberry Pi Tips + +### SD Card Wear + +Consider using tmpfs for `/var/log`: + +```bash +# Add to /etc/fstab +tmpfs /var/log tmpfs defaults,noatime,nosuid,size=100M 0 0 +``` + +### Network + +- Use static IP for stable NAS mounts +- Consider WiFi power management: `sudo iw dev wlan0 set power_save off` + +### Storage + +- External USB drives should be labeled consistently +- Use UUIDs in fstab for reliability: `fstab` with UUID mounts more reliable than device paths + +--- + +## Quick Start + +```bash +# 1. Update system +sudo apt update && sudo apt upgrade -y + +# 2. Install applications +sudo ./raspberrypi/rasp_pi_install_apps.sh + +# 3. Configure mounts +sudo ./raspberrypi/raspberry_pi_fstab.sh + +# 4. Reboot +sudo reboot +``` \ No newline at end of file diff --git a/readme.md b/readme.md index 0c8f4cd..a011e19 100644 --- a/readme.md +++ b/readme.md @@ -1,46 +1,264 @@ # Public Setup Files -This repository contains a collection of scripts and configuration snippets I use across various systems. Most scripts target Debian/Ubuntu Linux, but you'll also find utilities for macOS, Raspberry Pi, OpenWrt, and Docker environments. The goal is to speed up system deployment and automate routine maintenance tasks. +> Collection of scripts, configuration files, and documentation for the WickedYoda homelab. --- -## 🔎 Directory Overview +## Quick Start -- **Email_domains_block_allow/** – Lists of domains to allow or block for email filtering. -- **Kali_tools_install/** – Scripts to install popular Kali Linux security tools on Debian-based systems, including [Kali Purple setup instructions](https://wickedyoda.com/?p=3131). -- **batch_files/** – Windows batch files for backups and other automation. -- **blocked_domains/** – Domain blocklists used by some of the setup scripts. -- **cockpit/** – Installation helper for the Cockpit web administration interface. -- **copy-move-files/** – Shell scripts to copy or move media files between locations. -- **copy_palworld_local/** – Python helper for backing up Palworld server data. -- **cron-job-setup-files/** – Tools for configuring recurring cron jobs. -- **debian-files/** – Miscellaneous Debian utilities (e.g., installing bashtop). -- **docker/** – Docker and Docker Compose setups including Greenbone, Jellyfin and Portainer. -- **fstab-setup/** – Scripts to build and apply `/etc/fstab` mount entries. -- **git_clone_setup/** – Notes and helpers for cloning repositories. -- **mac-scripts/** – Assorted macOS helper scripts. -- **observium/** – Installer for the Observium monitoring agent. -- **openwrt_scripts/** – Utilities for managing OpenWrt routers. -- **paperless-ngx/** – Script to install the Paperless‑ngx document server. -- **raspberrypi/** – Backup and application install scripts for Raspberry Pi systems. -- **reset_perms/** – Simple scripts to reset file and directory permissions. -- **server_config/** – Example configuration files for Debian and Ubuntu servers. -- **snmp/** – SNMP configuration scripts for Linux and OpenWrt. -- **system_command_run/** – Run a command across multiple machines via SSH. -- **telegraf-setup-scripts/** – Install script for the Telegraf metrics agent. -- **ubuntu-based/** – Application install script for Ubuntu-based systems. -- **unblocked_domains_for_VPN/** – Domain list allowed through a VPN connection. -- **updates_scripts/** – Scripts to update or upgrade packages on a system. +### Clone and Set Up -Root-level helper scripts include: +```bash +# Automatic setup (installs git if needed) +curl -s https://raw.githubusercontent.com/wickedyoda/public-setupfiles/refs/heads/main/update-from_repo.sh | sudo bash -- `update-from_repo.py` – Python script to sync updates from this repository. -- `update-from_repo.sh` – Bash version of the update sync script. +# Or manual clone +git clone https://github.com/wickedyoda/public-setupfiles.git +cd public-setupfiles +chmod -R 755 . +``` + +### Update Scripts + +```bash +# Pull latest changes +./update-from_repo.sh + +# Or Python version +python3 update-from_repo.py +``` --- -## 🐙 Cloning This Repository +## 📚 Documentation -To clone this repository and switch to the main branch: +| Category | Description | +|----------|-------------| +| [Getting Started](.github/wiki/Getting-Started.md) | Clone, update, and basic usage | +| [Core Scripts](.github/wiki/Core-Scripts.md) | `update-from-repo`, system updates, log collection | +| [Docker Setup](.github/wiki/Docker-Setup.md) | Docker installation, compose examples, backups | +| [System Updates](.github/wiki/System-Updates.md) | Auto-updates, Kali updates, Debian upgrades | +| [Linux Setup](.github/wiki/Linux-Setup.md) | Debian/Ubuntu server config, fstab, permissions | +| [Monitoring](.github/wiki/Monitoring.md) | Uptime Kuma bot, SNMP, Telegraf, Observium | +| [Network](.github/wiki/Network.md) | WireGuard VPN, Pi-hole, domain lists | +| [NAS & Storage](.github/wiki/NAS-Storage.md) | SMB/CIFS mounts, backups, sync scripts | +| [Windows Batch Files](.github/wiki/Windows-Batch-Files.md) | Backup and automation scripts | +| [macOS Scripts](.github/wiki/macOS-Scripts.md) | Homebrew, system updates, window tiling | +| [Raspberry Pi](.github/wiki/Raspberry-Pi.md) | Pi setup, apps, mounts, media | +| [OpenWrt](.github/wiki/OpenWrt.md) | Router configuration | +| [Kali & Security](.github/wiki/Kali-Security.md) | Kali Linux setup, security tools | +| [Domain Lists](.github/wiki/Domain-Lists.md) | Blocklists, allowlists, filtering rules | +| [Troubleshooting](.github/wiki/Troubleshooting.md) | Common issues and solutions | -```sudo curl -s https://raw.githubusercontent.com/wickedyoda/public-setupfiles/refs/heads/main/update-from_repo.sh | sudo bash``` \ No newline at end of file +--- + +## 📁 Repository Structure + +``` +public-setupfiles/ +├── .github/ +│ ├── workflows/ +│ │ ├── codeql.yml # GitHub CodeQL security analysis +│ │ └── security-scans.yml # ShellCheck, pip-audit, Docker Scout +│ ├── SECURITY_SCANS.md # Security scanning documentation +│ └── secret_scanning.yml # Secret scanning configuration +│ +├── Docker_backup/ # Docker data backup scripts +│ ├── docker-backup.sh +│ ├── setup_docker-backup.sh +│ ├── correct_docker-backup.sh +│ └── update_docker-backup.sh +│ +├── Email_domains_block_allow/ # Email server domain policies +│ ├── blocked_domains.md +│ └── email_domain_allow_block_list.md +│ +├── Kali_tools_install/ # Kali Linux security tools +│ ├── Kalitoolsinstall.sh +│ └── setup_debian_kali_tools.sh +│ +├── WG VPN setup/ # WireGuard client setup +│ └── install-wireguard-client.sh +│ +├── batch_files/ # Windows .bat automation scripts +│ ├── Various backup/sync scripts +│ └── Older Files/ +│ +├── cockpit/ # Cockpit web admin installer +│ └── cockpit_install.sh +│ +├── copy-move-files/ # Media file transfer scripts +│ ├── Brandons-Minecraft_backup.sh +│ ├── media_to_usb.sh +│ ├── nas public.sh +│ └── public_to_usb.sh +│ +├── copy_palworld_local/ # Palworld server backup +│ └── copy_palworld_local.py +│ +├── cron-job-setup-files/ # Cron job configuration +│ ├── setup_cron_job_updates.sh +│ ├── setup_cron_job_updates-nomount.sh +│ └── CronUpdates-readme.txt +│ +├── debian-dist-upgrades/ # Debian version upgrade scripts +│ └── dist-upgrade.sh +│ +├── debian-files/ # Debian utility scripts +│ ├── install-bashtop.sh +│ └── update_tailscale.sh +│ +├── docker/ # Docker and Docker Compose setups +│ ├── install_docker_debian.sh +│ ├── install_docker_ubuntu.sh +│ ├── jellyfin-docker-compose.yml +│ ├── portainer/ +│ └── gluetun/ +│ +├── docker_stuff/ # Docker cleanup and utilities +│ ├── docker_cleanup_updates.sh +│ ├── install_docker_debian.sh +│ └── uptime_kurma_fix/ +│ +├── domains/ # Domain block/allow lists +│ ├── bamboo_domains.txt +│ ├── bypass_sites.txt +│ ├── social_media.txt +│ ├── streaming_domains_whitelist.txt +│ └── known_porn_domains.txt +│ +├── fstab-setup/ # NAS mount configuration +│ ├── Fstab entries.txt +│ ├── setup-fstab.sh +│ └── setup-fstab-remake-mnt.sh +│ +├── gather-logs/ # System diagnostic script +│ └── gather_logs.sh +│ +├── git_clone_setup/ # Git repository helpers +│ ├── Git_clone_local_checkout.md +│ └── README.md +│ +├── kali-purple/ # Kali Purple security platform +│ └── install_kali_purple.sh +│ +├── mac-scripts/ # macOS automation +│ ├── updates.sh +│ ├── system_updates.sh +│ ├── brewscripts/ +│ ├── disable_DS_store.sh +│ ├── DS_STore_destroy.sh +│ ├── rsync basic.sh +│ └── tile-windows/ +│ +├── observium/ # Observium monitoring agent +│ ├── custom_install.sh +│ └── observium_agent/ +│ +├── openwrt_scripts/ # OpenWrt router utilities +│ ├── openwrt_full-upgrade.sh +│ ├── installing_ipref.sh +│ └── openwrt_snmp_defaults.txt +│ +├── paperless-ngx/ # Document management system +│ └── paperless-ngx-install.sh +│ +├── pihole/ # Pi-hole DNS blocklists +│ ├── blocklist.txt +│ ├── whitelist.txt +│ └── combined_list.txt +│ +├── raspberrypi/ # Raspberry Pi scripts +│ ├── rasp_pi_install_apps.sh +│ ├── raspberry_pi_fstab.sh +│ ├── Media_to_ap-share.sh +│ └── public_to_usb.sh +│ +├── redirecting-desktop_and_documents/ # Windows folder redirection +│ └── redirectfiles.sh +│ +├── reset_perms/ # Permission reset utilities +│ ├── reset_perms.sh +│ └── reset_perms 1.sh +│ +├── server_config/ # Debian/Ubuntu server templates +│ ├── debian/ +│ └── ubuntu/ +│ +├── snmp/ # SNMP configuration scripts +│ ├── setup_snmp.sh +│ ├── setup_snmp_openwrt.sh +│ ├── setup_snmp_openwrt_v2.sh +│ ├── setup_snmp_openwrt_publicv2.sh +│ └── Setup_snmp_macos.sh +│ +├── system_command_run/ # Distributed SSH command execution +│ ├── Run_command_on_machines.sh +│ ├── run_command_on_machines.py' +│ ├── run_command_on_machines_Keys.py' +│ └── machines.txt +│ +├── telegraf-setup-scripts/ # Telegraf metrics agent +│ ├── install-client.sh +│ └── update-telegraf-w-keys.sh +│ +├── ubuntu-based/ # Ubuntu application installer +│ └── ubuntu_install_apps.sh +│ +├── unblocked_domains_for_VPN/ # VPN bypass domain list +│ └── unblocked_domains.txt +│ +├── updates_scripts/ # System update automation +│ ├── updates.sh +│ ├── updates.py +│ ├── kali-updates.sh +│ ├── docker-run-netdata-install.sh +│ └── fix_apt_influx.sh +│ +├── uptime-updates/ # Uptime Kuma container sync bot +│ ├── uptime-bot.py +│ ├── config.yml +│ ├── container-monitor-map.yaml +│ ├── requirements.txt +│ └── setup-update_uptime-bot.sh +│ +├── LICENSE # MIT License +├── SECURITY.md # Security policy +├── collect_info.sh # System information collector +├── setup_cron_clean.sh # Set up automated updates +├── update-from_repo.py # Python repo sync script +├── update-from_repo.sh # Bash repo sync script +└── updates_clean.sh # Hardened update script for cron +``` + +--- + +## 🛡️ Security + +- All scripts are included **as-is** without warranty +- Review any script before running +- Some scripts require root/sudo privileges +- Never commit secrets or credentials +- See [SECURITY.md](SECURITY.md) for details + +--- + +## 🤝 Contributing + +1. Fork the repository +2. Create a feature branch (`git checkout -b feature/amazing-script`) +3. Commit your changes (`git commit -m 'Add amazing script'`) +4. Push to the branch (`git push origin feature/amazing-script`) +5. Open a Pull Request + +--- + +## 📄 License + +MIT License — see [LICENSE](LICENSE) file. + +--- + +## 📞 Contact + +For issues or questions, open an issue on [GitHub](https://github.com/wickedyoda/public-setupfiles/issues). \ No newline at end of file diff --git a/redirecting-desktop_and_documents/README.md b/redirecting-desktop_and_documents/README.md index dbd0101..813e944 100644 --- a/redirecting-desktop_and_documents/README.md +++ b/redirecting-desktop_and_documents/README.md @@ -1,40 +1,48 @@ # redirecting-desktop_and_documents -<<<<<<< HEAD -## Purpose -This directory contains files related to **redirecting desktop and documents**. - -## Contents -- `redirectfiles.sh` - -## Usage -- Review each script or configuration file before running -- Some scripts may require **root or sudo** -- Paths and variables may need adjustment for your environment - -## Notes -- This folder is part of the **public-setupfiles** repository -- Files are provided as-is for reference or automation - -## Safety -Always back up data before running scripts that modify system state. -======= ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **redirecting-desktop_and_documents**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Script to redirect Windows Desktop and Documents folders to OneDrive. --- ->>>>>>> main + +## Files + +| File | Description | +|------|-------------| +| `redirectfiles.sh` | Redirect Desktop/Documents to OneDrive | + +--- + +## Usage + +Run as Administrator on Windows: + +```batch +# Double-click or run from Command Prompt +redirectfiles.sh +``` + +--- + +## What It Does + +1. Creates backup of existing folders +2. Creates symbolic links to OneDrive equivalents +3. Moves existing content to OneDrive + +--- + +## Prerequisites + +- OneDrive must be installed and configured +- Desktop/Documents must not be in cloud sync yet +- Run as Administrator + +--- + +## Safety + +- Creates backups: `~/Desktop.backup`, `~/Documents.backup` +- If symlinks already exist, skips the operation +- Test on a non-critical folder first \ No newline at end of file diff --git a/reset_perms/README.md b/reset_perms/README.md index b5d959d..1e3927a 100644 --- a/reset_perms/README.md +++ b/reset_perms/README.md @@ -1,43 +1,59 @@ -<<<<<<< HEAD - # reset_perms - - ## Purpose - This directory contains files related to **reset perms**. - - ## Contents - - `reset_perms 1.sh` -- `reset_perms.sh` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # reset_perms ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **reset_perms**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Scripts to reset file and directory permissions on mounted storage. --- ->>>>>>> main + +## Files + +| File | Description | +|------|-------------| +| `reset_perms.sh` | Reset permissions on specified mounts | +| `reset_perms 1.sh` | Alternative version | + +--- + +## Usage + +```bash +sudo ./reset_perms/reset_perms.sh +``` + +--- + +## What It Does + +1. Creates mount directories if missing +2. Mounts specified devices +3. Creates bind mounts for timeshift +4. Sets permissions (777) on all paths + +--- + +## Paths Reset + +| Path | Device | +|------|--------| +| `/mnt/public` | /dev/sdc1 | +| `/mnt/public-bk` | /dev/sde1 | +| `/mnt/public2` | /dev/sdd1 | +| `/mnt/public2-bk` | /dev/sdb1 | +| `/mnt/timeshift` | Bind from public2 | +| `/mnt/timeshift1` | Bind from public2-bk | + +--- + +## Permissions + +- Owner: `root:root` +- Permissions: `777` (world read/write/execute) + +--- + +## Notes + +- Check device IDs match your system +- Modify mount commands if devices differ +- Run after system update or hardware change \ No newline at end of file diff --git a/server_config/README.md b/server_config/README.md index d5f0093..7da6bd3 100644 --- a/server_config/README.md +++ b/server_config/README.md @@ -1,40 +1,95 @@ # server_config -<<<<<<< HEAD -## Purpose -This directory contains files related to **server config**. - -## Contents -- (No files detected) - -## Usage -- Review each script or configuration file before running -- Some scripts may require **root or sudo** -- Paths and variables may need adjustment for your environment - -## Notes -- This folder is part of the **public-setupfiles** repository -- Files are provided as-is for reference or automation - -## Safety -Always back up data before running scripts that modify system state. -======= ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **server_config**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Server configuration templates and scripts for Debian/Ubuntu servers. --- ->>>>>>> main + +## Files + +| Directory | Description | +|-----------|-------------| +| `debian/` | Debian server setup scripts | +| `ubuntu/` | Ubuntu server setup scripts | + +--- + +## Debian Setup + +### `debian/debian-setup.sh` + +Initial Debian server configuration: + +```bash +sudo ./server_config/debian/debian-setup.sh +``` + +**Configures:** +- sudo access for user 'traver' +- open-vm-tools for VMware +- Git and wget +- Cockpit web UI +- Python + +--- + +## Ubuntu Setup + +### `ubuntu/ubuntu-setup.sh` + +Initial Ubuntu server configuration: + +```bash +sudo ./server_config/ubuntu/ubuntu-setup.sh +``` + +**Configures:** +- sudo access +- UFW removal (if desired) +- open-vm-tools +- Git and wget +- Cockpit web UI +- Python + +--- + +### `ubuntu/ubuntu-docker-install.sh` + +Docker installation on Ubuntu: + +```bash +sudo ./server_config/ubuntu/ubuntu-docker-install.sh +``` + +**Installs:** +- Docker CE + CLI + containerd +- Docker Compose plugin +- Adds user to docker group + +--- + +## First-Time Setup + +```bash +# 1. Run server setup +sudo ./server_config/ubuntu/ubuntu-setup.sh +# OR +sudo ./server_config/debian/debian-setup.sh + +# 2. Install Docker (optional) +sudo ./server_config/ubuntu/ubuntu-docker-install.sh + +# 3. Set up updates +sudo ./setup_cron_clean.sh # Auto-updates every 6h +``` + +--- + +## Post-Installation + +After running these scripts: +1. Log out and back in (for group changes) +2. Access Cockpit at `https://server-ip:9090` +3. Configure Docker if installed +4. Review auto-update settings in `/etc/cron.d/auto_updates` \ No newline at end of file diff --git a/snmp/README.md b/snmp/README.md index 8cd8d34..e600b77 100644 --- a/snmp/README.md +++ b/snmp/README.md @@ -1,46 +1,97 @@ -<<<<<<< HEAD - # snmp - - ## Purpose - This directory contains files related to **snmp**. - - ## Contents - - `setup_snmp_openwrt_v2.sh` -- `setup_snmp_openwrt_publicv2.sh` -- `setup_snmp_openwrt.sh` -- `setup_snmp.sh` -- `Setup_snmp_macos.sh` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # snmp ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **snmp**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Scripts for setting up SNMP (Simple Network Management Protocol) monitoring on various platforms. --- ->>>>>>> main + +## Files + +| Script | Platform | Version | +|--------|----------|---------| +| `setup_snmp.sh` | Linux | SNMPv3 | +| `setup_snmp_openwrt.sh` | OpenWrt | SNMPv3 | +| `setup_snmp_openwrt_v2.sh` | OpenWrt | SNMPv3 (alternative) | +| `setup_snmp_openwrt_publicv2.sh` | OpenWrt | SNMPv2 (public) | +| `Setup_snmp_macos.sh` | macOS | SNMPv3 | + +--- + +## Linux SNMP Setup + +### `setup_snmp.sh` + +```bash +sudo ./snmp/setup_snmp.sh +``` + +**What it does:** +1. Prompts for SNMPv3 credentials +2. Installs snmpd and snmp packages +3. Configures SNMPv3 with authPriv (MD5+AES) +4. Sets up firewall rules +5. Starts and enables SNMP service + +**Configures:** +- `/etc/snmp/snmpd.conf` +- Access restricted to local network + +--- + +## OpenWrt SNMP Setup + +### SNMPv3 + +```bash +ssh root@10.0.84.1 +# Run on router +curl -sSL https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/snmp/setup_snmp_openwrt.sh | sh +``` + +**Interactively prompts for:** +- SNMPv3 username +- Password +- System location +- System contact + +--- + +## macOS SNMP Setup + +### `Setup_snmp_macos.sh` + +```bash +# Install dependencies first +brew install net-snmp + +# Then run script +sudo ./snmp/Setup_snmp_macos.sh +``` + +--- + +## SNMP Monitoring + +### Test Connection + +```bash +# From remote system +snmpwalk -v2c -c public 10.0.84.1 system +# Or SNMPv3 +snmpwalk -v3 -u admin -l authPriv -a MD5 -A "password" \ + -x AES -X "password" 10.0.84.1 system +``` + +### Grafana Integration + +Use any Prometheus/Grafana SNMP exporter with the configured community strings. + +--- + +## Security + +- Use SNMPv3 with authPriv whenever possible +- Restrict SNMP to management network +- Change default community strings +- Do not expose to the internet \ No newline at end of file diff --git a/system_command_run/README.md b/system_command_run/README.md index 51ba37b..3c7baf7 100644 --- a/system_command_run/README.md +++ b/system_command_run/README.md @@ -1,45 +1,81 @@ -<<<<<<< HEAD - # system_command_run - - ## Purpose - This directory contains files related to **system command run**. - - ## Contents - - `machines.txt` -- `run_command_on_machines_Keys.py'` -- `Run_command_on_machines.sh` -- `run_command_on_machines.py'` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # system_command_run ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **system_command_run**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Run commands across multiple machines via SSH using a machine list file. --- ->>>>>>> main + +## Files + +| File | Description | +|------|-------------| +| `Run_command_on_machines.sh` | Bash script for SSH command execution | +| `run_command_on_machines.py'` | Python script (alternate, may have bugs) | +| `run_command_on_machines_Keys.py'` | SSH key-based version | +| `machines.txt` | Machine list (host, user, password) | + +--- + +## Usage + +### Bash Script + +```bash +# Edit machines.txt with target hosts +# Format: HOST USER PASSWORD (password can be empty for key auth) +./system_command_run/Run_command_on_machines.sh +``` + +**Prompt:** +``` +Enter command to execute: +``` + +**Example:** +``` +sudo apt update +``` + +--- + +## Configuration + +### machines.txt + +Format: `HOST USER PASSWORD` + +``` +docker1 root +docker2 root YOURPASSWORD +192.168.1.100 admin PASSWORD +``` + +Leave password empty to use SSH keys. + +--- + +## Security Notes + +- **Never commit machines.txt with real passwords** +- Use SSH keys for better security +- Restrict machine list to trusted hosts +- Keys version (`run_command_on_machines_Keys.py'`) uses key-based auth + +--- + +## Alternative: Python Keys Version + +```bash +python3 system_command_run/run_command_on_machines_Keys.py' +``` + +Uses paramiko with key-based SSH authentication. + +--- + +## Related + +- Check disk usage across machines +- Restart services on multiple hosts +- Run maintenance scripts remotely \ No newline at end of file diff --git a/telegraf-setup-scripts/README.md b/telegraf-setup-scripts/README.md index d4178b5..b029b2e 100644 --- a/telegraf-setup-scripts/README.md +++ b/telegraf-setup-scripts/README.md @@ -1,43 +1,71 @@ -<<<<<<< HEAD - # telegraf-setup-scripts - - ## Purpose - This directory contains files related to **telegraf setup scripts**. - - ## Contents - - `install-client.sh` -- `update-telegraf-w-keys.sh` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # telegraf-setup-scripts ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **telegraf-setup-scripts**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Install and configure Telegraf metrics agent for system monitoring. --- ->>>>>>> main + +## Files + +| Script | Description | +|--------|-------------| +| `install-client.sh` | Install Telegraf on Linux client | +| `update-telegraf-w-keys.sh` | Update with SSH key support | + +--- + +## Installation + +### Install Client + +```bash +curl -s https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/telegraf-setup-scripts/install-client.sh | sudo bash +``` + +Or download: + +```bash +curl -s https://raw.githubusercontent.com/wickedyoda/public-setupfiles/main/telegraf-setup-scripts/install-client.sh -o install-telegraf.sh +chmod +x install-telegraf.sh +sudo ./install-telegraf.sh +``` + +--- + +## What It Does + +The script installs: +1. Telegraf service +2. Default configuration (often with InfluxDB input/output) +3. Optional: Secure secret storage + +--- + +## Configuration + +Telegraf configuration is typically at: +- `/etc/telegraf/telegraf.conf` (Linux) +- Custom configs via flags + +Edit to add your InfluxDB or Prometheus endpoint. + +--- + +## Uninstallation + +```bash +# Remove package +sudo apt remove telegraf +sudo apt autoremove + +# Remove config +sudo rm -rf /etc/telegraf +``` + +--- + +## Related + +- [Monitoring Wiki](../Monitoring.md) +- `observium/` — Alternative monitoring agent \ No newline at end of file diff --git a/ubuntu-based/README.md b/ubuntu-based/README.md index f7cc314..ea9b049 100644 --- a/ubuntu-based/README.md +++ b/ubuntu-based/README.md @@ -1,40 +1,70 @@ # ubuntu-based -<<<<<<< HEAD -## Purpose -This directory contains files related to **ubuntu based**. - -## Contents -- `ubuntu_install_apps.sh` - -## Usage -- Review each script or configuration file before running -- Some scripts may require **root or sudo** -- Paths and variables may need adjustment for your environment - -## Notes -- This folder is part of the **public-setupfiles** repository -- Files are provided as-is for reference or automation - -## Safety -Always back up data before running scripts that modify system state. -======= ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **ubuntu-based**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Scripts for Ubuntu-based system setup and application installation. --- ->>>>>>> main + +## Files + +| Script | Description | +|--------|-------------| +| `ubuntu_install_apps.sh` | Install common applications on Ubuntu | + +--- + +## Usage + +```bash +sudo ./ubuntu-based/ubuntu_install_apps.sh +``` + +--- + +## What It Installs + +The script installs a curated set of applications commonly needed on Ubuntu systems: + +### Development Tools +- git, curl, wget + +### System Utilities +- htop (system monitoring) +- tmux (terminal multiplexer) + +### Networking +- openssh-server +- nmap (network scanner) + +### Media +- VLC media player +- ffmpeg (video/audio tools) + +### Productivity +- KeepassXC (password manager) +- LibreOffice (office suite) + +--- + +## Requirements + +- Ubuntu 20.04+ recommended +- Root/sudo access +- Internet connection + +--- + +## Post-Installation + +After running the script: +1. Log out and back in for group changes +2. Review installed applications +3. Configure any tools that need customization + +--- + +## Related + +- `debian-files/` — For Debian systems +- `updates_scripts/` — For system updates \ No newline at end of file diff --git a/unblocked_domains_for_VPN/README.md b/unblocked_domains_for_VPN/README.md index 09fcd24..c58dd61 100644 --- a/unblocked_domains_for_VPN/README.md +++ b/unblocked_domains_for_VPN/README.md @@ -1,40 +1,63 @@ # unblocked_domains_for_VPN -<<<<<<< HEAD -## Purpose -This directory contains files related to **unblocked domains for VPN**. - -## Contents -- `unblocked_domains.txt` - -## Usage -- Review each script or configuration file before running -- Some scripts may require **root or sudo** -- Paths and variables may need adjustment for your environment - -## Notes -- This folder is part of the **public-setupfiles** repository -- Files are provided as-is for reference or automation - -## Safety -Always back up data before running scripts that modify system state. -======= ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **unblocked_domains_for_VPN**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +A list of domains that should bypass VPN connections for proper functionality. --- ->>>>>>> main + +## Files + +| File | Description | +|------|-------------| +| `unblocked_domains.txt` | Domains allowed through VPN | + +--- + +## Usage + +### VPN Configuration + +Add these domains to your VPN client's "split tunnel" or "bypass" settings: + +```bash +# Example: OpenVPN route directive +route domain.com net_gateway +``` + +Or in WireGuard: + +```ini +[Peer] +AllowedIPs = 10.0.0.0/8 +Endpoint = twy4us.duckdns.org:51820 +PersistentKeepalive = 25 +``` + +--- + +## Domain Categories + +### Personal Sites + +- `wickedyoda.com` — Personal website +- `tyates.one` — Personal domain +- `twy4.us` — Short URL service + +### Work Sites + +- `traverates.com` — Work domain +- `sampyates.com` — Business domain +- `designsbymantha.com` — Design portfolio + +### Internal/Cloud + +- `homeassistant.local` — Home automation +- Cloud storage domains (personal) + +--- + +## Related + +- `pihole/` — DNS blocking for VPN-connected hosts +- `domains/` — General domain lists \ No newline at end of file diff --git a/updates_scripts/README.md b/updates_scripts/README.md index 2c7b16d..586a0ec 100644 --- a/updates_scripts/README.md +++ b/updates_scripts/README.md @@ -1,44 +1,101 @@ -<<<<<<< HEAD - # updates_scripts - - ## Purpose - This directory contains files related to **updates scripts**. - - ## Contents - - `updates.py` -- `kali-updates.sh` -- `updates.sh` - - ## Usage - - Review each script or configuration file before running - - Some scripts may require **root or sudo** - - Paths and variables may need adjustment for your environment - - ## Notes - - This folder is part of the **public-setupfiles** repository - - Files are provided as-is for reference or automation - - ## Safety - Always back up data before running scripts that modify system state. -======= # updates_scripts ## Overview -This directory is part of the **public-setupfiles** repository. -It contains scripts, configuration files, or resources related to **updates_scripts**. -## Usage -- Review scripts before execution -- Some files may require **root or sudo privileges** -- Paths, variables, or credentials may need customization - -## Logging & Output -Scripts may generate logs or output files in the same directory or system locations. - -## Safety Notes -- Always back up important data before running scripts -- Test in a non-production environment when possible -- Use at your own risk +Automated system update scripts for various platforms and use cases. --- ->>>>>>> main + +## Scripts + +| File | Description | Platform | +|------|-------------|----------| +| `updates.sh` | Standard update script | Linux | +| `updates.py` | Python version of updates.sh | Linux | +| `kali-updates.sh` | Kali Linux rolling update with tool selection | Kali/Debian | +| `docker-run-netdata-install.sh` | Install Netdata in Docker | Docker host | +| `fix_apt_influx.sh` | Repair InfluxDB APT repository | Debian | +| `dist-upgrade.sh` | Debian distro upgrade wrapper | Debian | + +--- + +## Standard Updates + +### `updates.sh` + +```bash +sudo ./updates.sh +``` + +**Commands:** +```bash +apt update +apt upgrade -y +apt full-upgrade -y +apt autoremove -y +apt clean -y +apt purge -y +``` + +--- + +## Kali Updates + +### `kali-updates.sh` + +```bash +sudo ./kali-updates.sh +``` + +**Features:** +- Updates Kali rolling release +- Interactive tool selection +- Optional Kali Purple installation + +--- + +## Docker Netdata + +### `docker-run-netdata-install.sh` + +```bash +sudo ./docker-stuff/docker_run-netdata-install.sh +``` + +Creates a Netdata container for host metrics monitoring. + +--- + +## InfluxDB Fix + +### `fix_apt_influx.sh` + +```bash +sudo ./updates_scripts/fix_apt_influx.sh +``` + +**Fixes:** +- InfluxDB repo key verification +- Sury PHP repo setup +- Stale APT metadata cleanup + +--- + +## Dist Upgrade + +### `dist-upgrade.sh` + +Interactive tool for moving to newer Debian releases. + +```bash +sudo ./debian-dist-upgrades/dist-upgrade.sh +``` + +See `debian-dist-upgrades/` for details. + +--- + +## Related Scripts + +- `setup_cron_clean.sh` (repo root) — Schedules these scripts +- `updates_clean.sh` (repo root) — Hardened cron version \ No newline at end of file diff --git a/uptime-updates/README.md b/uptime-updates/README.md index 0bdf4f1..c2f0b48 100644 --- a/uptime-updates/README.md +++ b/uptime-updates/README.md @@ -1,94 +1,128 @@ -# Uptime Updates Bot +# Uptime Updates -This directory contains a bot that keeps Uptime Kuma Docker monitor container IDs in sync when containers are recreated. +Uptime Kuma container monitoring synchronization bot. + +--- + +## Overview + +The uptime-updates bot keeps Docker container IDs synchronized with Uptime Kuma monitors when containers are recreated. ## Files -- `config.yml` - Uptime Kuma URL, API key, and bot behavior settings. -- `container-monitor-map.yaml` - Docker host/container to Uptime monitor ID mappings. -- `uptime-bot.py` - Continuous sync bot runner. -- `setup-update_uptime-bot.sh` - Installer/updater for Linux systemd. -- `requirements.txt` - Python dependencies used by the bot virtualenv. +| File | Description | +|------|-------------| +| `uptime-bot.py` | Main Python bot | +| `config.yml` | Uptime Kuma API configuration | +| `container-monitor-map.yaml` | Container-to-monitor mappings | +| `requirements.txt` | Python dependencies | +| `setup-update_uptime-bot.sh` | Installation script | +| `export-watchtower-bundle.sh` | Create Docker bundle | -## Requirements +## Installation -- Linux host with systemd -- Root or sudo access -- Docker remote API reachable for each mapped host (`host_api`) +### Using Setup Script -## Configure +```bash +sudo bash uptime-updates/setup-update_uptime-bot.sh +``` -1. Edit `config.yml`: - - Set `uptime_kuma.url` - - Set `uptime_kuma.api_key` - - Optional: adjust `bot.poll_interval_seconds` +### Options -2. Edit `container-monitor-map.yaml`: - - Add one mapping per container - - Set `host_api`, `container_name`, and `uptime_monitor_id` +```bash +sudo bash setup-update_uptime-bot.sh \ + --install-dir /opt/uptime-updates \ + --service-name uptime-updates-bot.service \ + --user root +``` -## Install And Start Service +--- -Run from this directory: +## Configuration +### config.yml + +```yaml +uptime_kuma: + url: "http://127.0.0.1:3001" + api_key: "YOUR_API_KEY_HERE" + +bot: + dry_run: false + request_timeout_seconds: 10 + poll_interval_seconds: 60 +``` + +### container-monitor-map.yaml + +```yaml +mappings: + - name: "media-server jellyfin" + enabled: true + host_api: "tcp://10.0.84.21:2375" + container_name: "jellyfin" + uptime_monitor_id: 21 + expected_uptime_type: "docker" +``` + +--- + +## Docker Standalone Deployment + +See `uptime-updates/watchtower-export/` for Docker Compose files: +- `docker-compose.watchtower-uptime-sync.yml` +- `docker-compose.watchtower-embedded.yml` + +--- + +## Usage with Docker Remote API + +### Enable Docker TCP API + +```bash +# On each Docker host +sudo bash /path/to/public-setupfiles/docker_stuff/setup-docker-remote-api.sh --allow-ip 10.0.84.50 +``` + +### Update Monitor Mapping + +Edit `container-monitor-map.yaml` and re-run: ```bash sudo bash setup-update_uptime-bot.sh ``` -What this does: -- Installs/updates files under `/opt/uptime-updates` -- Ensures compatible Python is available (for `uptime-bot.py`) -- Creates virtualenv and installs dependencies from `requirements.txt` -- Writes/updates systemd service -- Enables service on boot and restarts it +The bot reloads configuration each cycle, so most changes apply without restart. -## Update After Changes +--- -After editing files in this directory, re-run: +## Watchtower Export + +### Files + +| File | Purpose | +|------|---------| +| `Dockerfile` | Bot container image | +| `requirements.txt` | Python deps | +| `docker-compose.watchtower-uptime-sync.yml` | Separate containers | +| `docker-compose.watchtower-embedded.yml` | Embedded mode | + +### Build Bundle ```bash -sudo bash setup-update_uptime-bot.sh +cd uptime-updates +sudo bash export-watchtower-bundle.sh ``` -The installer is idempotent and acts as both installer and updater. +Creates timestamped tar.gz for sharing. -By default on update: -- Existing live `/opt/uptime-updates/config.yml` is preserved -- Existing live `/opt/uptime-updates/container-monitor-map.yaml` is preserved +--- -Use overwrite flags only when you want to replace live files from repo copies. - -## Common Commands +## Monitoring ```bash -systemctl status uptime-updates-bot.service -journalctl -u uptime-updates-bot.service -f -systemctl restart uptime-updates-bot.service -systemctl stop uptime-updates-bot.service -systemctl disable uptime-updates-bot.service -``` +# Check service status +systemctl status uptime-updates-bot -## Installer Options - -```bash -sudo bash setup-update_uptime-bot.sh --help -``` - -Useful options: -- `--install-dir PATH` -- `--service-name NAME` -- `--user USER` -- `--group GROUP` -- `--no-enable` -- `--no-start` -- `--min-python-version 3.9` -- `--overwrite-config` -- `--overwrite-map` -- `--overwrite-all` -- `--dry-run` - -## Notes - -- If using Docker TCP API (`tcp://host:2375`), secure network access appropriately. -- `expected_uptime_type: docker` in mapping items helps prevent accidental updates to non-docker monitors. -- The bot reloads `config.yml` and `container-monitor-map.yaml` each cycle, so most config changes do not require service restart. +# View logs +journalctl -u uptime-updates-bot -f +``` \ No newline at end of file