- Changed input field IDs and names from 'dest_endpoint' and 'source_endpoint' to 'dest_host' and 'source_host' for consistency across NextCloud and WebDAV forms.
- Updated placeholder texts for username, password, and file path fields to be more user-friendly.
- Enhanced the URL parsing and validation logic in the transfer configuration generation to ensure proper handling of WebDAV and NextCloud URLs, improving robustness and error handling.
- Updated the Login template to conditionally display external authentication options based on the presence of enabled providers.
- Modified the HandleLoginPage and HandleLogin functions to check for available external providers and pass this information to the Login template.
- Improved user experience by dynamically showing or hiding external sign-in options during the login process.
- Introduced two new migrations (011b_recover_notification_services_rename and 011c_recover_auth_providers_rename) to check and recover the notification_services and auth_providers tables if they were incorrectly renamed during a previous migration.
- Implemented logic to handle various states of the tables, including logging warnings for potential issues and ensuring safe recovery actions.
- Updated the migration sequence to include these new recovery steps, enhancing the robustness of the database migration process.
- Introduced a new migration (011a_recover_transfer_configs_rename) to check and recover the transfer_configs table if it was incorrectly renamed to _transfer_configs_old during a previous migration.
- Implemented logic to handle various states of the tables, including logging warnings for potential issues.
- Ensured that the migration can be safely added to the existing migration sequence.
This migration enhances the robustness of the database migration process by addressing potential inconsistencies.
- Changed the logic in the HandleDeleteUser function to check for other administrators using the actual 'is_admin' column instead of counting all admins.
- Prevented deletion if no other administrators exist, ensuring at least one admin remains in the system.
https://github.com/StarFleetCPTN/GoMFT/issues/67
- Introduced a new migration (013_cleanup_invalid_booleans) to update boolean columns represented as integers in the transfer_configs, notification_services, and auth_providers tables.
- Ensured that only valid values (0, 1, or NULL) are retained, setting any invalid values to 0.
- Added error handling for potential issues during the migration process, including checks for table existence before attempting updates.
This migration improves data integrity by standardizing boolean representations across the database.
https://github.com/StarFleetCPTN/GoMFT/issues/69
- Enhanced the logic for updating user and group IDs by deleting and recreating the user/group if the current IDs do not match the specified values.
- Added verification steps to ensure the UID/GID changes were successful, with appropriate error handling and logging for failures.
- Streamlined the process for handling Alpine Linux user management, improving clarity and maintainability of the script.
https://github.com/StarFleetCPTN/GoMFT/issues/68
- Removed manual parsing and validation of cron expressions, relying on the cron instance's AddFunc for validation.
- Enhanced error logging to provide more informative messages when scheduling fails, including the original schedule string.
- Streamlined the scheduling process by using the original schedule directly, improving code clarity and maintainability.
https://github.com/StarFleetCPTN/GoMFT/issues/71
- Added logic to drop any existing temporary tables from previous failed migration runs before renaming the current table.
- Enhanced error logging to provide warnings if the drop operation fails, ensuring smoother migration execution.
https://github.com/StarFleetCPTN/GoMFT/issues/72
- Added a hidden input for the notification form to ensure proper handling of the 'is_enabled' state.
- Updated the configuration loading to allow skipping SSL verification with a new environment variable.
- Introduced temporary log file creation for rclone commands, improving error handling and logging of file processing.
- Enhanced the transfer executor to parse rclone logs for file metadata, ensuring accurate tracking of processed files.
- Added debug logging for received form data in the notification service update handler to aid in troubleshooting.
- Updated notification service models to use pointers for boolean fields, allowing for better handling of enabled states.
- Introduced helper methods for getting and setting enabled states in notification and auth provider models.
- Refactored notification form templates to consolidate event trigger handling, improving code maintainability.
- Added SSL verification configuration to the application settings, allowing users to skip SSL verification for outgoing notifications.
- Enhanced the notification sending logic to respect the SSL verification setting.
- Introduced new migration to alter boolean defaults in the database schema for better compatibility with the updated models.
- Added unit tests for email and Rclone service functionalities to ensure reliability.
- Integrated new components for job scheduling, including a cron-based scheduler and improved job execution logic.
- Refactored job executor to utilize interfaces for better dependency management and testing.
- Enhanced notification system with a new Notifier interface, allowing for flexible notification service integration.
- Added comprehensive unit tests for job executor, scheduler, and notification components to ensure reliability and correctness.
- Updated metadata handling to support new database interfaces, improving testability and maintainability.
- Introduced a new endpoint to test connections for source and destination providers.
- Implemented a TestResult component to display success or error messages based on connection outcomes.
- Enhanced the configuration form with buttons to initiate connection tests, improving user experience.
- Updated backend logic to handle connection testing and return appropriate feedback to the frontend.
- Refactored existing handlers and routes to accommodate the new testing functionality.
- Added fields for pre-rendering flags in the config form, allowing for better user experience during edits.
- Updated Rclone command options to accept current command ID for pre-selection.
- Introduced region input fields for MinIO source and destination forms, enhancing configuration flexibility.
- Modified backend logic to handle region parameters when generating Rclone configurations.
- Improved flag handling by parsing selected flags and values for better state management in the UI.
- Added sorting capabilities to file metadata lists, including SortBy and SortDir fields.
- Integrated a toast notification system for user feedback on actions.
- Updated templates and JavaScript for improved user interaction and responsiveness.
- Refactored file metadata handlers to utilize new sorting features and ensure proper data flow.
- Introduced new notification dialog components for better user confirmation on actions.
- Added new types for file metadata filtering and data structures.
- Created templates for displaying file metadata details, lists, and search functionalities.
- Implemented JavaScript utilities for handling modals and notifications related to file metadata actions.
- Enhanced the user interface with Tailwind CSS for better styling and responsiveness.
- Introduced pagination and search capabilities for file metadata management.
- Changed default port initialization for source and destination configurations to 0, triggering default settings based on connection type.
- Implemented logic to set default ports for FTP (21) and SFTP (22) in the frontend templates.
- Enhanced input fields for FTP and SFTP forms to initialize and update port values dynamically based on user selections.
- Updated backend configuration generation to include the correct port values for source and destination settings.
- Added a new build script using esbuild to bundle JavaScript and CSS assets, including Tailwind CSS and Font Awesome.
- Updated the Dockerfile to include a multi-stage build process for frontend assets, ensuring efficient image creation.
- Enhanced GitHub workflows to automate the installation of Node.js dependencies and build frontend assets during CI/CD.
- Introduced a .gitignore entry for the dist directory to prevent unnecessary files from being tracked.
- Improved caching and content type handling for static files served by the application.
- Introduced a new struct for NotificationService within the notification form data, encompassing various configuration fields for services like Pushbullet, Ntfy, Gotify, and Pushover.
- This addition enhances the form's capability to manage detailed service configurations, improving user experience and flexibility in notification management.
- Removed unused dependencies from go.mod and go.sum, including testify and objx.
- Expanded the README to include detailed information on authentication providers and user management features, enhancing clarity on security and integration options.
- Organized user management section to better outline authentication options and security features.
- Deleted outdated test files for providers, authentication, password management, configuration, database, email, scheduler, and web handlers.
- Cleaned up the codebase by removing tests that are no longer relevant or have been replaced by new implementations.
- This cleanup helps maintain a more manageable and efficient testing environment.
- Implemented a modal to display detailed information for audit logs.
- Added event listeners for showing and hiding the modal based on user interactions.
- Enhanced user experience by allowing users to view JSON-formatted details in the modal.
- Included functionality to close the modal when clicking outside of it.
- Added new fields for Pushbullet, Ntfy, Gotify, and Pushover services, including API keys, device identifiers, and message templates.
- Updated the notification form to dynamically display existing values for configuration fields.
- Enhanced backend handlers to support the retrieval of additional service configuration data.
- Improved user experience by ensuring that the form reflects current settings for each notification service type.
- Added a confirmation dialog for deleting notification services to prevent accidental deletions.
- Updated the notifications management interface to include improved delete button functionality.
- Enhanced JavaScript event handling for delete requests, providing user feedback on success and error scenarios.
- Refactored notification service deletion logic to ensure accurate tracking and messaging during deletion operations.
- Added new routes and handlers for creating and editing notification services.
- Implemented a dedicated page for adding new notification services with a comprehensive form.
- Updated the notifications management interface to include links for editing existing services.
- Enhanced backend logic to support the retrieval and updating of notification service configurations.
- Improved user experience with toast notifications for success and error messages during service management.
- Implemented Pushbullet, Ntfy, Gotify, and Pushover notification services in the notifications management system.
- Enhanced the user interface to include configuration fields for each new service type, including API keys, device identifiers, and message templates.
- Updated backend logic to handle sending notifications through the newly integrated services, including error handling for API responses.
- Added event trigger options for job start, completion, and error notifications for each service.
- Improved test notification functionality to verify configurations for all supported services.
- Added a new notifications management page with functionality to create, delete, and test notification services.
- Introduced a new template for displaying notification services, including forms for adding and configuring services.
- Updated backend handlers to support notification service operations and improved error handling.
- Enhanced the user interface with toast notifications for success and error messages related to notification actions.
- Refactored existing settings components to integrate with the new notifications management features.
- Introduced new templates for displaying and managing user notifications, including pagination support.
- Implemented helper functions for formatting notification times and determining icon and background colors based on notification types.
- Enhanced the user interface with controls for marking notifications as read and adjusting the number of notifications displayed per page.
- Added a dropdown for quick access to notifications and a count badge for unread notifications.
- Included a help section to guide users on managing their notifications effectively.
- Added new templates for managing authentication providers, including forms for creating and editing providers.
- Implemented backend logic to handle retrieval, creation, and deletion of authentication providers.
- Introduced new database migrations to support the storage of authentication provider data.
- Enhanced the user interface to display available authentication providers and their statuses.
- Added routes and handlers for managing authentication provider actions in the web application.
- Added functionality to manage command flags and their values in the configuration form.
- Updated the database schema to include a new field for storing command flag values.
- Enhanced backend logic to process and store flag values for non-boolean flags during configuration creation and updates.
- Improved the user interface to dynamically require destination fields based on selected commands.
- Refactored related templates and handlers to support the new command flag management features.
- Added support for selecting rclone commands and their associated flags in the configuration form.
- Implemented backend logic to handle rclone command retrieval and flag management.
- Enhanced the database schema to include rclone command and flag fields in the transfer configuration.
- Created new migration scripts to add rclone-related tables and fields.
- Updated the scheduler to execute transfers using the selected rclone command and flags.
- Developed new handlers for rendering rclone command options and flags in the web interface.
- Improved error handling and logging for rclone command execution and configuration updates.
- Added toast notifications for user actions, improving user feedback on operations such as creation, deletion, and updates.
- Implemented a confirmation dialog for user deletions, ensuring users are prompted before irreversible actions.
- Refactored user management templates to streamline the user interface and enhance usability.
- Improved error handling for user operations, providing clearer messages for success and failure scenarios.
- Updated JavaScript functions to manage user interactions and modal displays more effectively.
- Updated user management templates to improve user experience with a new layout and success/error message handling.
- Implemented password hashing for user creation and management, enhancing security.
- Refactored dashboard components to include better data presentation and responsiveness.
- Added dark mode support for various components, ensuring a consistent user interface across themes.
- Improved toast notification animations for better user feedback during actions.
- Implemented pagination for user notifications, allowing users to navigate through their notifications more efficiently.
- Updated the notifications page to display the total count of notifications and provide controls for adjusting the number of notifications shown per page.
- Added backend support for fetching paginated notifications and counting total notifications for users.
- Improved the user interface for notifications, including a new layout and additional help sections to guide users on managing their notifications.
- Added duplication feature for configurations and jobs, allowing users to create copies of existing entries.
- Introduced new buttons in the UI for duplicating configurations and jobs, enhancing user experience.
- Implemented backend logic to handle duplication requests, ensuring proper ownership checks and data integrity.
- Added notifications for successful duplication actions to inform users of the outcome.
- Updated relevant templates and handlers to support the new duplication functionality.
- Upgraded `golang.org/x/crypto` to v0.36.0 and other indirect dependencies to their latest versions.
- Added functionality to assign admin roles to users upon creation in the main application logic.
- Introduced new templates for admin role management, including forms for creating and editing roles.
- Removed deprecated admin tools template to streamline the admin interface.
- Added audit logging for role changes to improve tracking and accountability.
- Updated the AdminTools template to format JSON payloads for better readability.
- Modified the HandleAdminTools function to create a template context with user information before rendering the AdminTools view.
- Added functionality to allow users to select and reorder job configurations in both new and edit job forms.
- Introduced JavaScript logic to handle the display and ordering of selected configurations, including move up/down buttons.
- Updated backend to process and store the order of configurations when creating or updating jobs.
- Enhanced logging for job creation and update processes to include configuration order details.
- Introduced a new section for testing webhook notifications within the admin tools.
- Added a form to input webhook URL, secret, custom headers, and payload for testing.
- Implemented backend logic to handle webhook test requests and display results.
- Created a toast notification component to show success or failure messages for webhook tests.
- Updated the AdminToolsData structure to include fields for webhook test results.
- Introduced a new section in the admin tools for testing email configurations.
- Added a form to send test emails, including recipient, subject, and message fields.
- Implemented backend logic to handle test email requests and send emails using the configured SMTP server.
- Created a toast notification component to display success or failure messages for email tests.
- Updated routes to include a new endpoint for handling test email submissions.
- Added debug logging for job scheduling, execution, and configuration processing.
- Included detailed log messages for cron expression validation and file transfer commands.
- Improved visibility into job history and webhook notifications with additional debug information.
- Added support for build arguments in Dockerfile to specify UID, GID, version, build time, and commit hash.
- Updated docker-compose.yaml to utilize UID and GID for non-root user execution, improving file permission handling.
- Introduced entrypoint.sh script to manage user permissions and environment setup at container startup.
- Enhanced application logging to include version information during startup.
- Updated README with new instructions for running the application with specific user IDs and environment variables.
- Added configuration options for Two-Factor Authentication, including TOTP encryption key.
- Updated user profile components to manage 2FA status and backup codes.
- Implemented encryption and decryption of TOTP secrets using AES-256-GCM.
- Enhanced backup code generation and validation processes, storing hashed codes securely.
- Introduced new routes and handlers for managing backup codes and 2FA setup.
- Updated README with detailed instructions and security considerations for 2FA implementation.
- Added support for Two-Factor Authentication, including setup, verification, and disabling features.
- Introduced new components for 2FA setup and verification, enhancing user security.
- Updated user model to include fields for 2FA secret, status, and backup codes.
- Created migration to add necessary database fields for 2FA.
- Enhanced authentication handlers to manage 2FA processes during login and profile updates.
- Updated routes to include 2FA-related endpoints for setup and verification.
- Deleted the migration that updated the use_builtin_auth field to separate source and destination fields.
- Introduced a new migration to update source_type and destination_type from 'google_drive' to 'gdrive' in transfer_configs.
- Remove renaming logic for job_histories and password_histories tables, ensuring consistency with plural naming.
- Adjust CREATE TABLE statements to reflect the correct plural forms for job_histories and password_histories.
- Update drop table order to match the new naming conventions, maintaining foreign key constraints.
- Replace the previous warning message with a more concise and clear statement regarding the active development status of GoMFT.
- Emphasize the potential for rapid changes in configurations and the importance of reviewing release notes before updates.
- Separate built-in authentication options for source and destination in configuration forms.
- Update UI components to reflect changes in authentication handling for Google Drive and Google Photos.
- Implement path validation for local directories in the configuration forms, enhancing user experience.
- Revise README to clarify supported source and destination types, including updated terminology for Google Drive.
- Introduce new API endpoint for path validation, improving error handling and user feedback.
- Change background and text colors in configuration forms to enhance visibility and consistency with dark mode.
- Update button classes and layout for better responsiveness and user experience.
- Revise authentication prompts and information sections for Google Drive and Google Photos to clarify the authentication process.
- Ensure all text elements are styled appropriately for both light and dark themes, improving overall accessibility.
- Add a new logo for GoMFT in SVG and PNG formats.
- Center the application title in the README and enhance the layout with a warning message about active development.
- Update the home page template to display the new logo prominently.
- Revise the screenshots section to improve visual presentation and clarity.
- Add Google Drive and Google Photos as supported storage options in the features section.
- Include Google OAuth configuration parameters in the environment variable section.
- Update transfer configuration options to specify Google Photos and Google Drive specific settings.
- Improve documentation clarity for file options and performance settings.
- Introduce Google Photos as a source and destination option in the configuration forms.
- Implement authentication handling and configuration management for Google Photos.
- Update database schema and migrations to include new fields related to Google Photos.
- Enhance UI components to support Google Photos-specific settings and forms.
- Add tests for Google Photos integration, ensuring proper functionality and authentication flow.
- Limit concurrent transfers for Google Photos to ensure compliance with API restrictions.
- Add Google Drive as a source and destination option in the configuration forms.
- Implement Google Drive authentication flow and token management.
- Update job and configuration handlers to support Google Drive-specific settings.
- Enhance UI components to include Google Drive configuration templates.
- Introduce new tests for Google Drive integration and ensure proper handling of authentication and configuration.
- Update database migrations to accommodate new fields related to Google Drive configurations.
- Remove unnecessary print statements for form data and configuration details in the HandleCreateConfig function.
- Update handling of skipProcessedFiles to use a pointer for improved clarity and functionality.
- Introduce a comprehensive testing guide for the GoMFT application, detailing testing strategies, types, and best practices.
- Add new test utilities and examples for various components, including unit, integration, API, and webhook tests.
- Update the maximum concurrent transfers setting in the UI from 20 to 32 to support increased transfer capabilities.
- Add webhook notification settings to job configuration, allowing users to enable notifications for job success and failure.
- Implement webhook payload structure and authentication using HMAC-SHA256 for secure communication.
- Enhance the admin tools with a log viewer and system management features, including database backup and log file access.
- Update README documentation to include details on webhook integration and admin tools.
- Introduce comprehensive tests for webhook functionality, ensuring correct payload delivery and header validation.
- Add database migrations to support new webhook fields in job configurations.
- Update .gitignore to include new provider files and ensure proper tracking.
- Refactor job and config forms to support multiple configuration selections, improving user experience.
- Implement logic to handle the initialization of skipProcessedFiles with a default value.
- Add new provider form templates for better organization and management of source and destination configurations.
- Enhance tests for provider forms and job configurations to ensure robust functionality.
- Introduce nullable handling for skipProcessedFiles in the database schema and update related migrations.
- Improve error handling and validation in job creation and editing processes.
- Add support for multiple transfer configurations per job, allowing users to select one or more configurations.
- Update job creation and editing forms to handle multiple configuration selections with checkboxes.
- Enhance job processing logic to iterate through all associated configurations during execution.
- Introduce database migrations to add necessary fields for storing multiple configuration IDs.
- Update dashboard and history views to display configuration details for jobs.
- Refactor related templates and handlers to accommodate the new multi-configuration functionality.
- Introduce new test file for providers, covering integration and rendering of source and destination components
- Implement tests for source and destination provider availability
- Enhance scheduler tests with new scenarios for job unscheduling, log rotation, and file processing history
- Add tests for output pattern processing with date and filename variables
- Ensure accessibility and dynamic rendering of provider forms in tests
- Add new dependencies: `github.com/joho/godotenv`, `github.com/stretchr/testify`, and `gopkg.in/natefinch/lumberjack.v2`
- Remove indirect dependency on `github.com/joho/godotenv`
- Refactor JobRunDetails template to separate content rendering for improved testing
- Enhance error message display in JobRunDetails template
- Introduce new test files for JWT and password functionalities
- Add comprehensive tests for database operations and error handling
- Add new log viewer component in admin tools
- Support dynamic log file browsing and content display
- Implement custom scrollbar for log content
- Add log file refresh, download, and view capabilities
- Update environment variables for log configuration
- Enhance logging system with more flexible configuration options
- Implement concurrent file transfer processing with configurable concurrency
- Add new `max_concurrent_transfers` column to transfer configurations
- Enhance scheduler to support multi-threaded file transfers
- Introduce advanced logging system with rotation and configurable log levels
- Update Docker Compose and documentation with new logging configuration options
- Modify directory structure to separate data, backups, and logs
- Add environment variables for comprehensive logging control
- Improve error handling and logging in file transfer processes
- Introduce new `skip_processed_files` option for transfer configurations
- Add database migration to support the new column
- Update config form and UI to include skip processed files toggle
- Enhance scheduler to respect skip processed files setting
- Modify file processing logic to optionally skip previously processed files
- Add environment variable support for skip processed files configuration
- Add new section describing file metadata tracking features
- Include screenshot for file metadata view
- Expand feature list with comprehensive metadata management capabilities
- Update user guide with instructions for managing file metadata
- Remove Font Awesome icons from status badges in components
- Improve layout and styling for mobile responsiveness
- Implement HandleFileMetadataSearchPartial handler for HTMX-powered search results
- Update file_metadata.templ to use new /files/search/partial endpoint
- Add HTMX request detection and browser redirect for search results
- Enhance search results pagination and filtering with partial rendering
- Improve user experience with dynamic, asynchronous search interactions
- Rename FileMetadataListContent to FileMetadataListPartial for consistency
- Add separate FileMetadataSearchContent template for search results
- Enhance handler logic to support HTMX partial rendering
- Implement more robust HTMX request detection
- Improve error handling and empty state rendering for file lists and search results
- Implement partial rendering templates for file metadata list and search
- Add HTMX attributes for dynamic pagination and filtering
- Create loading indicators for HTMX requests
- Enhance user experience with smooth, asynchronous content updates
- Improve pagination and filtering interactions using HTMX
- Replace separate rclone size and list commands with single lsjson operation
- Improve file metadata extraction from JSON output
- Add support for hash and file size retrieval from remote sources
- Optimize file processing logic with more robust metadata handling
- Enhance error handling and logging for file listing and transfer processes
- Add mechanism to track processed files within a single job run
- Ensure unique file processing by filtering out duplicate entries
- Improve logging for skipped duplicate files
- Enhance file transfer reliability and efficiency
- Add x-init hook to set default form values in config form
- Adjust scrollbar hover color in layout
- Create marker files for providers, source, and destination packages
- Remove unnecessary error styling in local source form
- Create provider-specific form templates for source and destination storage types
- Implement common form fields and type selection components
- Support multiple storage providers with consistent form design
- Update .gitignore to exclude generated provider files
- Comment out OneDrive and Google Drive options in source and destination storage type selects
- Remove corresponding form templates for OneDrive and Google Drive authentication
- Temporarily disable these cloud storage options from the configuration interface
- Update executeJob method to handle source and destination paths for S3, MinIO, and B2
- Add bucket support for size, list, transfer, and archive operations
- Ensure correct path construction for different storage types
- Remove duplicate error logging in archive operation
- Update config_form.templ to include form fields for new storage types
- Modify db.go to support generating rclone configs for WebDAV, NextCloud, OneDrive, and Google Drive
- Add migration to include new cloud storage fields in transfer_configs table
- Update migrations.go to include new cloud storage migration
- Replace mattn/go-sqlite3 with glebarez/sqlite for better cross-platform compatibility
- Add gormigrate for database migration support
- Update go.mod and go.sum with new dependencies
- Modify database initialization to use the new SQLite driver
- Ignore compiled binary in .gitignore
- Implement mobile navigation with bottom nav and hamburger menu
- Add mobile-specific meta tags for PWA and iOS compatibility
- Enhance CSS for mobile responsiveness and safe area insets
- Improve JavaScript for mobile UX with responsive tables and form enhancements
- Configure Air.toml for development workflow
StarFleetCPTN