🚀 go-pugleaf

Big Mama 01 @ 🇬🇧 UK - London:
Host: onlyservers.com

CPU: 2x Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz [2x (10c/20t)]
RAM: 256G DDR4 ECC
BOOT: 2x 480G SSD
DISK: 2x 8T SSD
NET: 25 Gbit @ 1000 TB monthly

📥 That will boost import performance to the moon! 🎯


Archive Nodes @ 🇫🇮 FI - Helsinki:
Host: pulsedmedia.com

1x Front Node / Overview
CPU: Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz
RAM: 16G DDR4
DISK: 2T NVME
NET: 1 Gbit

2x Storage Nodes
CPU: Intel(R) Core(TM) i5-6500T CPU @ 2.50GHz
RAM: 16G DDR4
DISK: 4T NVME
NET: 1 Gbit

Moving data now.
                    

💥 AMAZING NEWS! We've received a $250 dedicated server as a physical donation:

🖥️  6-month Xeon-E31240, 16GB DDR3 ECC, 4×8TB HDD - Located in Europe! 🇪🇺

🌍 Newsgroup Migration started:

📰 rocksolid.* and more → 🇺🇸 rocksolid-us.pugleaf.net (full retention ~1990-2025)

+ The New Home for:
++ RockSolidBBS.com / NovaLink.us
++ NovaBBS.com / NovaBBS.org
++ i2pn2.org


Moving newsgroups to another Server:

🔄 Migration Status: 🇩🇪 pi-dach.dorfdsl.de is restored and will be available again soon!
🤝 Community: We hope Marco will consider hosting a little pugleaf node!

📦 Recovery:
The pi-dach backup is running here and is (or should be) up to date.
+ Cronjob runs every 5 minutes to fetch new articles. We'll see how this works!

📰 de.* and ger.* → 🇩🇪 dorfdsl.pugleaf.net (full retention ~1990-2025)

📰 pl.* → 🇵🇱 polish.pugleaf.net  (full retention ~1990-2025)

📰 gmane.* → gmane.pugleaf.net (news.gmane.io)

📰 gwene.* → gwene.pugleaf.net (news.gmane.io)


📍 Status of Newsgroups Migration:

    ➤ Visit Statistics to monitor ongoing import progress

    ➤ Article counts (should) update every 5 minutes

🐛 Community Support: Found bugs or have suggestions?
Please post to newsgroup: 'rocksolid.nodes.help'
We welcome all feedback and your input helps us improve the system!
Your suggestions are valuable to us.

📧 Note: Email support is not currently available.

⚡ Early Launch: This service went live ahead of schedule!
                    

Welcome to our new high-performance NNTP newsgroup system built with Go.

**🚀 ANNOUNCEMENT: First go-pugleaf NNTP Node Online - Carrying Forward Retro Guy's Legacy**

We are proud to announce that the first **go-pugleaf** NNTP node is now online and operational!

**🌐 Access Information:**
- **Web Interface:** rocksolid-us.pugleaf.net
- **NNTP Server:** Coming within the next few days
- **Coverage:** Full rocksolid.* hierarchy + additional groups

**📊 Current Status:**
✅ **ONLINE** - Web interface fully functional
🔄 **BACKFILLING** - Importing articles from legacy rslight installations
📥 **FETCHING** - Automated article retrieval via cron jobs

**🛠️ Development Roadmap:**
- **Next Few Days:** NNTP server for reading access
- **In Progress:** Incoming article peering setup
- **TODO:** Posting access and full bidirectional peering

**💝 Tribute to Retro Guy:**
This project carries forward the legacy of **Retro Guy** @novabbs.com
The visionary lead developer of rslight.
Though he sadly cannot see this evolution of his work... RIP!
We are committed to keeping the rocksolid.* community alive and thriving.

The fresh, modern interface and robust Go-based architecture represent
our dedication to honoring his contributions while building for the future.

**🔧 Technical Highlights:**
- Built with modern Go architecture for performance and reliability
- High-speed parallel article fetching and processing
- Efficient database design for large-scale article storage
- Full NNTP protocol compliance

**🤝 Community Notes:**
We're in the exciting phase of bringing historical rocksolid.* content back online!
The migration from legacy rslight installations is proceeding smoothly, ensuring no community history is lost.

**🎯 Mission:**
To provide a stable, modern, and efficient platform for the rocksolid.* community and beyond!

Stay tuned for NNTP access announcements and peering updates!

**The go-pugleaf Development Team**
*Honoring the past, building the future*

We are not responsible for this spam in news.admin.peering...
---

*Cross-posted to relevant groups.
For technical updates,visit the web interface.
                    

### Core NNTP Server Architecture
- [x] **Server Foundation**:
- [x] Main NNTP server struct with configuration
- [x] TCP listener with TLS/SSL support
- [x] Client connection management
- [x] Connection pooling and resource limits
- [x] Graceful shutdown handling
- [x] Multi-threaded request processing

### NNTP Protocol Implementation (RFC 3977)
- [x] **Essential Commands (Priority 1)**:
- [x] `CAPABILITIES` - Server capability announcement
- [x] `MODE READER` - Switch to reader mode
- [x] `AUTHINFO USER/PASS` - User authentication
- [x] `QUIT` - Clean connection termination
- [x] `HELP` - Command help system

- [x] **Group Management Commands**:
- [x] `LIST` - List available newsgroups
- [x] `LIST ACTIVE` - Active groups with counts
- [x] `LIST NEWSGROUPS` - Groups with descriptions
- [x] `GROUP` - Select newsgroup and get stats
- [x] `LISTGROUP` - List article numbers in group

- [x] **Article Retrieval Commands**:
- [x] `STAT` - Check article existence
- [x] `HEAD` - Retrieve article headers
- [x] `BODY` - Retrieve article body
- [x] `ARTICLE` - Retrieve complete article
- [x] Support for both message-ID and article number addressing

- [x] **Overview Commands**:
- [x] `XOVER` - Article overview data (NOV format)
- [x] `XHDR` - Specific header retrieval
- [ ] `HDR` - RFC 3977 header command
- [ ] `OVER` - RFC 3977 overview command

- [ ] **Advanced Commands (Priority 2)**:
- [x] `POST` - Article posting (optional)
- [x] `IHAVE` - Article transfer (server-to-server)
- [x] `CHECK/TAKETHIS` - Streaming commands
- [ ] `DATE` - Server date/time
- [ ] `NEWGROUPS` - New groups since date
- [ ] `NEWNEWS` - New articles since date

### Database Integration
- [x] **Data Access Layer**:
- [x] Integrate with existing `*database.Database`
- [x] Reuse `models.Article`, `models.Newsgroup` structures
- [x] Efficient article retrieval by message-ID and number
- [x] Group statistics and article counting
- [x] XOVER data generation from database
- [x] Range-based overview queries for XOVER/XHDR
- [x] Article caching for frequently accessed content

### Authentication & Security
- [x] **User Authentication**:
- [x] Integrate with existing user system (`models.User`)
- [x] AUTHINFO command with database user validation (framework)
- [x] Session management for authenticated connections
- [x] Permission-based group access control

- [ ] **Security Features**:
- [ ] Rate limiting per connection/IP
- [ ] Command validation and sanitization
- [ ] Logging of all NNTP operations
- [ ] Connection timeout management
- [ ] IP-based access control (optional)

### Configuration & Management
- [x] **Server Configuration**:
- [x] NNTP server port and binding configuration
- [x] TLS/SSL certificate management
- [x] Maximum connections and resource limits
- [ ] Command-specific feature toggles
- [x] Integration with existing `config.ServerConfig`

- [ ] **Administration Integration**:
- [ ] NNTP server status in admin dashboard
- [ ] Real-time connection monitoring
- [ ] Command usage statistics
- [ ] Server performance metrics
- [ ] Configuration management via web interface

### Testing & Validation
- [ ] **Protocol Compliance**:
- [ ] RFC 3977 compliance testing
- [ ] Test with standard NNTP clients (tin, slrn, Thunderbird)
- [ ] Multi-line response handling
- [ ] Error code consistency
- [ ] Character encoding support (UTF-8)

---
                    

### Admin Interface
- [x] Create admin dashboard
- [x] API Tokens Management
- [ ] Add server status monitoring
- [x] Implement newsgroup management:
- [x] Add new groups
- [x] Delete groups
- [x] Modify group settings
- [x] sections Create/Set/Modify for newsgroups
- [x] Bulk operations
- [x] Create user management interface
- [x] Add provider configuration
- [x] Implement system settings
- [ ] Create backup/restore tools
- [ ] Implement backup automation

### Monitoring & Logging
- [ ] Create health check endpoints
- [ ] Create log rotation
- [ ] Add alerting system

### Maintenance Tools
- [x] Create article cleanup tools
- [x] Add database optimization
- [x] Create migration scripts

---
                    

### User Management ✅ COMPLETE
- [x] Implement user registration with validation
- [x] Add password hashing (bcrypt)
- [x] Create login/logout system
- [x] Implement session management
- [x] Add JWT token support
- [x] Create user profile system
- [x] Profile: password change functionality
- [ ] Add password reset functionality via email
- [ ] Add external SMTP sending support
- [ ] Implement email verification for new accounts (+ option in admin panel do enable/disble)
- [x] Implement user roles/permissions
- [x] Add user registration form template
- [x] Create login/logout page templates
- [x] **Admin interface for user management (create/update/delete)**
- [x] **API token management system with CRUD operations**
- [x] **Session-based web authentication with secure cookies**

### Search Functionality ⚡ IN PROGRESS
- [x] **Basic search page template implemented**
- [x] **Search API endpoints (newsgroup search working)**
- [x] Create search API endpoints
- [x] **Newsgroup name search functionality**
- [ ] Implement full-text search for articles (FTS5 schema ready)
- [x] Add search page template
- [x] Add advanced search filters (author, date, group)
- [x] Implement search result pagination
- [ ] Add search result highlighting

### Security Features ✅ PARTIALLY COMPLETE
- [x] **API token authentication system with secure hashing**
- [x] **Secure session handling with HttpOnly cookies**
- [x] **Password validation and bcrypt hashing**
- [x] **Input validation framework**
- [x] **SQL injection prevention (parameterized queries)**
- [x] **XSS protection via template HTML escaping**
- [x] Implement SQL injection prevention (already in place with parameterized queries)
- [ ] Add CSRF protection
- [ ] Implement rate limiting middleware
- [ ] Add comprehensive input validation framework
- [ ] Add file upload security
- [ ] Create audit logging

### Theme & UI Enhancements ✅ COMPLETE
- [x] **Bootstrap 5.3 dark mode integration**
- [x] **Modern theme switcher with localStorage persistence**
- [x] **Responsive design optimized for mobile**
- [x] **Professional navigation with authentication links**
- [x] **Theme selector positioned bottom-right to avoid overlap**
- [x] **Page refresh functionality for complete theme application**

---
                    

### Gin Web Framework Setup
- [x] Set up Gin web server (internal/web/)
- [x] Create router configuration
- [x] Add paginated API endpoints:
- [x] GET /api/v1/groups (list newsgroups)
- [x] GET /api/v1/groups/:group/overview (group overview)
- [x] GET /api/v1/groups/:group/articles/:articleNum (single article)
- [x] GET /api/v1/groups/:group/threads (thread data)
- [x] Create cmd/web/main.go server entry point
- [x] Add middleware:
- [x] Logging middleware
- [x] Static file serving
- [x] CORS middleware
- [x] Error handling middleware
- [x] Create comprehensive error handling system
- [x] Add request validation

### 🔥 REVOLUTIONARY FORUM-STYLE THREADING SYSTEM
- [x] **Clean thread list page**: Shows only thread titles with message counts
- [x] **Full thread view route**: `/groups/:group/thread/:threadRoot`
- [x] **Complete forum experience**: Click thread → see full discussion
- [x] **Full article content inline**: All messages display complete article bodies
- [x] **Flat threading hierarchy**: Forum-style (no confusing nested replies)
- [x] **Smart threading engine**: Parses Message-ID/References for perfect grouping
- [x] **Chronological message sorting**: Root first, then replies by date
- [x] **Rich metadata display**: Author, date, lines, bytes for each message
- [x] **Thread navigation**: Clean breadcrumbs and back-to-threads links

### HTML Templates & UI
- [x] Create base template layout with theme support
- [x] Design newsgroup listing page with pagination
- [x] Create article display template with proper formatting
- [x] Implement pagination controls for Groups and Articles
- [x] **Revolutionary thread.html template with full article content**
- [x] **Updated threads.html for clean thread-only listing**
- [x] Create stats page with system information
- [x] Add help page with documentation
- [x] Design error page with user-friendly messages
- [x] Add comprehensive navigation
- [x] **Forum-style thread discussions with inline content**

### Static Assets & Frontend
- [x] Create modern CSS framework (responsive design)
- [x] Implement multiple theme system:
- [x] Modern theme (gradient design with animations)
- [x] Classic theme (retro terminal green-on-black)
- [x] Amber theme (vintage amber terminal)
- [x] Add JavaScript functionality:
- [x] Theme switching with localStorage persistence
- [x] Responsive navigation
- [x] Pagination controls
- [x] Article navigation
- [x] Mobile-friendly responsive design
- [x] RockSolid rocket logo integration

### Performance & Scalability
- [x] Implement efficient pagination for all large datasets
- [x] Optimize database queries with proper indexing
- [x] Memory-efficient template rendering
- [x] Preloader system for newsgroup descriptions
- [x] Reply count tracking and display
- [x] Error handling for missing articles/groups
- [x] **Full article loading optimization for thread views**
- [x] **Efficient thread message grouping and sorting**

---
                    

### Database Schema Design
- [x] Design main database schema
- [x] config table (system configuration)
- [x] providers table (NNTP server configurations)
- [x] newsgroups table (subscribed groups metadata)
- [x] users table (user accounts)
- [x] sessions table (user sessions)
- [x] user_permissions table (access control)
- [x] Design per-group database schemas
- [x] articles table (full article storage)
- [x] **threads table (revolutionary forum-style threading)**
- [x] overview table (XOVER data for performance)
- [x] Create database migration system
- [x] Implement database connection pooling

### SQLite3 Implementation
- [x] Create database abstraction layer (internal/database/)
- [x] Implement main database management
- [x] Implement per-newsgroup database management
- [x] Add article storage functions
- [x] **Implement revolutionary threading logic using Message-ID/References**
- [x] **Forum-style flat threading (one thread per original post)**
- [x] Create overview data management (XOVER caching)
- [x] Add database indexing for performance
- [x] Implement efficient batching and scalability features
- [x] **Thread counting and message grouping**
- [ ] Implement database backup/restore

### Data Models & Import System
- [x] Create Go structs for all data models
- [x] Implement JSON serialization
- [x] Add validation tags
- [x] Create model relationships
- [x] Add database query methods
- [x] Implement NNTP-to-database import pipeline:
- [x] Group subscription and metadata import
- [x] Article batch import using XOVER/XHDR
- [x] Threading logic based on Message-ID/References
- [x] Incremental sync for new articles
- [x] Downloaded article tracking system
- [x] Scalable batching for large groups
- [x] Batched sync for new articles
- [x] Downloaded article tracking system
- [x] Scalable batching for large groups

---