Restructure Directories: Organize Character Bios

Alex Johnson

-Oct 9, 2025

Restructure Directories: Organize Character Bios

🗂️ Reorganizing Directory Structure: Consolidating Character Information in `bios`

Hey guys! Let's talk about how we can make our project a whole lot cleaner and easier to manage. Currently, the way we store information about each character in our biographies is a bit scattered. This means more work for everyone and a higher chance of things going wrong. So, we're going to overhaul the directory structure to keep everything neat and tidy. Think of it as spring cleaning for our digital shelves!

😩 The Current Mess: Scattered Information

Right now, the data related to each character is spread across multiple directories, creating a bit of a headache. Imagine you're trying to find everything about Harry S. Truman. You'd have to rummage through different folders to piece together his story. Here’s a simplified look:

📁 bios/
├── harry_s_truman/
│   ├── chapter-01.md
│   ├── chapter-02.md
│   ├── ...
│   ├── The biography of Harry S. Truman.md
│   └── sources.md
│
📁 schemes/
├── harry_s_truman - sources.md
├── harry_s_truman - work plan.md
├── joseph_stalin - sources.md
├── joseph_stalin - work plan.md
└── winston_churchill - sources.md

This scattered approach leads to some serious problems:

❌ Inconsistency: Information gets duplicated across directories. Ugh!
❌ Maintenance Nightmare: Every change requires updating multiple locations. Double ugh!
❌ Confusing Navigation: Developers have to hunt around to find what they need. Frustrating!
❌ Risk of Desynchronization: Files can easily become outdated. Triple ugh!

🚀 The Proposed Solution: A Unified Approach

To fix this, we're going to consolidate all the information for each character into a single directory under bios/[normalized_name]/. This is the key to getting organized! Let's see what the new structure looks like:

📁 bios/
├── harry_s_truman/
│   ├── 📂 research/                    # ← Research information
│   │   ├── sources.md                 # ← Moved from schemes/
│   │   ├── work-plan.md         # ← Moved from schemes/
│   │   └── research-notes.md     # ← New: additional notes
│   ├── 📂 chapters/                   # ← Organized chapters
│   │   ├── chapter-01.md
│   │   ├── chapter-02.md
│   │   └── ...
│   ├── 📂 sections/                   # ← Special sections
│   │   ├── prologue.md
│   │   ├── introduction.md
│   │   ├── epilogue.md
│   │   ├── chronology.md
│   │   ├── glossary.md
│   │   ├── dramatis-personae.md
│   │   └── sources.md
│   ├── 📂 output/                     # ← Generated files
│   │   ├── The biography of Harry S. Truman.md
│   │   └── metadata.json
│   ├── 📂 control/                    # ← Control files (existing)
│   └── 📂 kdp/                        # ← Amazon KDP assets (existing)

This new setup will bring order to the chaos, making everything easier to find, update, and maintain.

📖 Detailed Structure: Breaking It Down

Let's dive into the specifics of each directory and file:

Main Directory: `bios/[normalized_name]/`

1. `research/` - Research Information

This is where all the research-related stuff goes. Think of it as the foundation of the biography. It includes:

research/
├── sources.md              # Collected and validated sources
├── work-plan.md      # Structured work plan
├── research-notes.md  # Additional notes from the process
└── validation-sources.json # Automatic validation results

2. `chapters/` - Book Chapters

All the chapters of the book will be organized here:

chapters/
├── chapter-01.md
├── chapter-02.md
├── ...
└── chapter-20.md

3. `sections/` - Special Sections

Special sections such as the prologue, introduction, and epilogue will be stored here:

sections/
├── prologue.md
├── introduction.md
├── epilogue.md
├── chronology.md
├── glossary.md
├── dramatis-personae.md
└── sources.md

4. `output/` - Final Generated Files

This directory will contain the final output, ready for publishing:

output/
├── The biography of [Character].md    # Complete biography
├── metadata.json                     # Work metadata
└── generation-log.txt                # Generation process log

5. `control/` and `kdp/` - Existing Directories

These directories will stay as they are, maintaining their existing functions:

control/     # Quality control and validation
kdp/         # Assets for Amazon KDP

🛠️ Migration Plan: Step-by-Step Guide

We're going to follow a phased approach to ensure a smooth transition:

Phase 1: Preparation

☐ Migration Script: Create a Python script to automate the file movement. We'll get to that in a bit.
☐ Backup: Make a complete backup before we start. Always a good idea!
☐ Validation: Verify the integrity of the existing files. Make sure everything's in good shape.

Phase 2: Restructuring

☐ Create New Structure: Generate directories based on the new schema.
☐ Move Files: Move files from the schemes/ directory to their new locations within the bios/ structure. For example:
- schemes/[character] - sources.md → bios/[character]/research/sources.md
- schemes/[character] - work plan.md → bios/[character]/research/work-plan.md
☐ Reorganize Chapters: Move chapters to the chapters/ subdirectory.
☐ Reorganize Sections: Move special sections to the sections/ subdirectory.

Phase 3: Code Update

☐ Application Paths: Update all references to file paths in the application.
☐ Generation Scripts: Adapt the scripts used for generating biographies to the new structure.
☐ Tests: Update any tests that rely on specific file paths.
☐ Documentation: Update the documentation to reflect the new organization.

Phase 4: Validation

☐ Integrity Tests: Verify that all files have been moved correctly.
☐ Functionality: Confirm that the generation process works as expected with the new structure.
☐ Cleanup: Remove the now-empty schemes/ directory. (Yay!)

💻 Affected Files and Components

Here are the files and components that will be affected by this change. This is just a heads-up so you know where to focus your attention.

Scripts to Create/Modify

# scripts/migrate_structure.py
def migrate_character_files(character_name):
    """Migrate files of a character to the new structure"""
    
# scripts/validate_migration.py  
def validate_file_integrity():
    """Validate that the migration was successful"""

Application Code

src/services/biography_generator.py: File paths.
src/services/file_manager.py: Directory management.
src/services/concatenation.py: File references.
src/worker/tasks.py: Celery tasks with file paths.

Configuration

config/settings.py: Path constants.
docker-compose.yml: Mounted volumes.
Tests: All tests that use specific file paths.

✅ Advantages of the New Structure: Why This Matters

This new organization isn't just about making things look pretty; it's about making our lives easier and our project more robust. Here's why:

🎯 Logical Organization

All information for a character in one place. Perfect!
Clear separation between research, content, and results.
Scalable structure for future characters. Ready for anything!

🔧 Simplified Maintenance

One directory per character for backups/restores. Sweet and simple.
Easy identification of missing or corrupted files.
Permission management per character.

🚀 Improved Development

Intuitive navigation in the IDE. Makes coding a breeze.
More efficient searches. Find what you need, fast!
Fewer errors due to incorrect paths. Goodbye, headaches!

📊 Enhanced Operations

Monitoring per character. Stay in the know.
Storage metrics per biography. Keep an eye on things.
Automated cleanup processes. Automatic win!

⚡ Proposed Migration Script: Let's Get Practical

Here's a basic Python script to get us started. We'll need to adapt this to our specific needs, but this gives you an idea of how the migration will work. Remember, back up your files first!

import os
import shutil
from pathlib import Path

def migrate_character_structure():
    """Migrate existing structure to new organization"""
    
    # Existing characters
    characters = ['harry_s_truman', 'joseph_stalin', 'winston_churchill']
    
    for character in characters:
        print(f"Migrating {character}...")
        
        # Create new structure
        base_path = Path(f"bios/{character}")
        (base_path / "research").mkdir(exist_ok=True)
        (base_path / "chapters").mkdir(exist_ok=True)
        (base_path / "sections").mkdir(exist_ok=True)
        (base_path / "output").mkdir(exist_ok=True)
        
        # Move files from schemes/
        source_files = Path("schemes").glob(f"{character} - *.md")
        for file in source_files:
            if "sources" in file.name:
                shutil.move(file, base_path / "research" / "sources.md")
            elif "work plan" in file.name:
                shutil.move(file, base_path / "research" / "work-plan.md")
        
        # Reorganize existing files in bios/
        # ... (reorganization logic)

✅ Acceptance Criteria: Ensuring Success

How will we know we've successfully completed this migration?

☐ Complete Migration: All files moved without any data loss.
☐ Functionality Preserved: The biography generation still works correctly.
☐ Updated Tests: All tests pass with the new structure.
☐ Updated Documentation: Guides reflect the new organization.
☐ Clean Code: References to the old structure are removed.
☐ Performance Maintained: No slowdown in processing times.

🏷️ Labels: Project Tags

refactor
file-structure
maintenance
organization

📅 Priority: High

This is a high-priority task because it significantly improves the project's organization and maintainability. It's like setting up a well-organized workspace – it makes everything run more smoothly.

⚠️ Risk Considerations: Heads Up!

Mandatory Backup: Always back up your files before starting the migration.
Thorough Validation: Validate file integrity after the move.
Rollback Plan: Have a plan to revert to the old structure if something goes wrong.
Collaboration: Coordinate with developers actively working on the project.

🔗 References: Where to Find More

Current structure in /bios and /schemes
Biography generation documentation
Existing file handling scripts

This refactoring will not only make our project easier to manage but also set us up for future growth. Great work, team! Let's get organized!

For more detailed information about file organization, you can visit Wikipedia.