BioGrids Site Installation Administration

BioGrids site installations provide a managed software stack for bioinformatics applications. When properly configured, new software is added, obsolete versions are removed, and updates happen automatically. This makes software updates and bug fixes available to users through simple requests to help@biogrids.org, all without requiring local system administration expertise.

Benefits of Site Installation Administration

Automated management: Software updates happen without manual intervention
Centralized control: Single point of management for entire research groups
Automatic cleanup: Obsolete software versions are removed automatically
Rolling updates: New software becomes available immediately upon release
Multi-platform support: Manage Linux and macOS software from a single system
Parallel operations: Efficient downloads and installations using multiple processes

Initial Site Setup

1. Create Dedicated Service Account

Create a dedicated biogrids user account for software management:

# On Linux
sudo adduser biogrids

# On macOS
sudo dscl . -create /Users/biogrids
sudo dscl . -create /Users/biogrids UserShell /bin/bash
sudo dscl . -create /Users/biogrids RealName "BioGrids Service Account"

Benefits of dedicated account: - Prevents accidental data loss during updates - Provides clear separation of software and user data - Enables proper permission management - Facilitates backup and maintenance procedures

2. Configure Installation Location

Choose an appropriate installation location:

# Switch to biogrids user
sudo su - biogrids

# Option 1: Use default location (requires sudo for initial creation)
# Installation will be at /opt/biogrids

# Option 2: Use custom location (recommended for large installations)
# Installation will be at specified path
export BIOGRIDS_TARGET="/data/biogrids"

3. Download and Setup BioGrids CLI

Download the appropriate version for your platform:

# For Linux
curl -LO https://biogrids.org/downloads/latest/biogrids-cli_linux.tar.gz
tar -zxf biogrids-cli_linux.tar.gz

# For macOS  
curl -LO https://biogrids.org/downloads/latest/biogrids-cli_macos.tar.gz
tar -zxf biogrids-cli_macos.tar.gz

4. Activate Site Installation

Perform initial site activation:

# With custom target directory (recommended)
./biogrids-cli activate-site --target /data/biogrids

# With default location
./biogrids-cli activate-site

During activation you'll be prompted for: - Site name (provided by BioGrids team) - Site key (provided by BioGrids team) - Installation preferences

The activation process will: - Create necessary directory structure - Install core BioGrids libraries - Configure automatic update checking - Set up initial cron tasks

Post-Installation Configuration

1. Create Required Symlinks

After activation, create the required /programs symlink on all machines that will use the software:

Linux Systems

# Create symlink (requires sudo)
sudo ln -sf /data/biogrids /programs

# Verify symlink
ls -la /programs

macOS Systems

On macOS Catalina and later, use the /etc/synthetic.conf method:

# Create synthetic.conf file
echo -e "programs\t/data/biogrids" | sudo tee /etc/synthetic.conf

# Reboot required for synthetic.conf to take effect
sudo reboot

2. Configure User Environment

Users need to configure their shell environment to use BioGrids software:

Bash Users

echo 'source /programs/biogrids.shrc' >> ~/.bashrc

Zsh Users (macOS default)

echo 'source /programs/biogrids.shrc' >> ~/.zshrc

Tcsh Users

echo 'source /programs/biogrids.cshrc' >> ~/.cshrc

3. Install Initial Software Collection

Install the complete BioGrids software collection:

# Install all versions of all software (recommended for sites)
biogrids-cli admin --all-versions

# Install only default versions (space-constrained environments)
biogrids-cli admin

Administration Commands

biogrids-cli admin Mode

The biogrids-cli admin command provides comprehensive site management:

Key functions: - Install all new software added to BioGrids - Update existing software when newer versions are available - Remove obsolete versions no longer supported - Maintain consistent software environment

Basic Administration Commands

Install All Software (Recommended)

# Complete installation with all versions
biogrids-cli admin --all-versions

# Multi-platform support (Linux + macOS)
biogrids-cli admin --all-versions --linux --darwin

# Parallel processing for faster installation
biogrids-cli admin --all-versions -j 8

Default Versions Only

# Install only default versions (saves disk space)
biogrids-cli admin

# With parallel processing
biogrids-cli admin -j 4

Platform-Specific Installation

# Linux software only
biogrids-cli admin --all-versions --linux

# macOS software only  
biogrids-cli admin --all-versions --darwin

# Both platforms (for NFS shared installations)
biogrids-cli admin --all-versions --linux --darwin

Automated Administration

Set Up Automatic Updates

Configure automatic updates using cron:

# Set up default crontab (checks every 15 minutes)
biogrids-cli crontab

# Set up with all versions (recommended)
biogrids-cli crontab --all-versions

# Multi-platform with parallel processing
biogrids-cli crontab --all-versions --linux --darwin -j 8

Manual Crontab Configuration

Edit crontab manually for custom schedules:

crontab -e

Example crontab entries:

# Complete administration every 15 minutes (recommended)
*/15 * * * * /programs/x86_64-linux/biogrids_installer/latest/biogrids-cli admin --all-versions -j 4

# Updates only (no new software installation)
*/15 * * * * /programs/x86_64-linux/biogrids_installer/latest/biogrids-cli update --all-titles --yes --quiet

# Daily cleanup (remove obsolete versions)
0 1 * * * /programs/x86_64-linux/biogrids_installer/latest/biogrids-cli clean --yes --quiet

# Weekly full synchronization
0 3 * * 0 /programs/x86_64-linux/biogrids_installer/latest/biogrids-cli admin --all-versions --linux --darwin -j 8

Maintenance Operations

Regular Maintenance Tasks

Check Installation Status

# View installed software
biogrids-cli list --long-format

# Check for available updates
biogrids-cli updates

# Verify installation integrity
biogrids-cli verify

Software Updates

# Update all installed software
biogrids-cli update --all-titles

# Update specific software
biogrids-cli update blast samtools

# Force reinstallation if needed
biogrids-cli update blast --reinstall

Cleanup Operations

# Remove obsolete software versions
biogrids-cli clean

# List what would be removed (dry run)
biogrids-cli clean --dry-run

# Force cleanup without confirmation
biogrids-cli clean --yes

Monitoring and Logging

Check System Status

# Network connectivity test
biogrids-cli check-connection

# System diagnostics
biogrids-cli diag

# Configuration verification
biogrids-cli verify

Log Management

# View installation logs
tail -f ~/.biogrids/logs/admin.log

# Check for errors
grep -i error ~/.biogrids/logs/*.log

# Monitor cron job output
tail -f /var/log/cron

Multi-User Management

User Access Control

Add Users to BioGrids Group

# Create biogrids group
sudo groupadd biogrids-users

# Add users to group
sudo usermod -a -G biogrids-users username1
sudo usermod -a -G biogrids-users username2

Configure Sudo Access (Optional)

# Allow users to run biogrids-cli as biogrids user
sudo tee /etc/sudoers.d/biogrids << EOF
%biogrids-users ALL=(biogrids) NOPASSWD: /programs/x86_64-linux/biogrids_installer/latest/biogrids-cli
EOF

User Environment Setup

System-wide Configuration

# Add to system-wide bashrc
echo 'source /programs/biogrids.shrc' | sudo tee -a /etc/bash.bashrc

# Add to system-wide profile
echo 'source /programs/biogrids.shrc' | sudo tee -a /etc/profile

Individual User Setup

# Script to configure new users
#!/bin/bash
# setup-biogrids-user.sh

USERNAME=$1
if [ -z "$USERNAME" ]; then
    echo "Usage: $0 <username>"
    exit 1
fi

# Add BioGrids to user's bashrc
echo 'source /programs/biogrids.shrc' >> /home/$USERNAME/.bashrc

# Add user to biogrids group
sudo usermod -a -G biogrids-users $USERNAME

echo "User $USERNAME configured for BioGrids access"

Troubleshooting Administration Issues

Common Administrative Problems

Disk Space Issues

# Check available disk space
df -h /data/biogrids

# Clean up obsolete versions
biogrids-cli clean

# Remove specific old versions
biogrids-cli remove software@old_version

Permission Problems

# Fix ownership of BioGrids installation
sudo chown -R biogrids:biogrids /data/biogrids

# Fix permissions for group access
sudo find /data/biogrids -type d -exec chmod 755 {} \;
sudo find /data/biogrids -type f -exec chmod 644 {} \;

Network and Update Issues

# Test network connectivity
biogrids-cli check-connection

# Force metadata refresh
biogrids-cli update-metadata

# Retry failed installations
biogrids-cli admin --all-versions --retry-failed

Performance Optimization

Optimize Download Performance

# Use parallel downloads
biogrids-cli config set parallel-downloads 8

# Configure download cache
biogrids-cli config set cache-size 20GB

# Use local mirror (if available)
biogrids-cli config set mirror-server local.biogrids.org

Optimize Storage Usage

# Install only essential software collections
biogrids-cli install -c "Genomics" -c "Proteomics"

# Use compression for archived versions
biogrids-cli config set compress-archives true

# Regular cleanup schedule
echo "0 2 * * * biogrids-cli clean --yes --quiet" | crontab -

Migration and Backup

Backup Procedures

Configuration Backup

# Backup BioGrids configuration
tar -czf biogrids-config-backup.tar.gz ~/.biogrids/

# Backup installation metadata
biogrids-cli save > biogrids-installation-config.json

Full Installation Backup

# Backup complete installation (requires significant storage)
rsync -av /data/biogrids/ backup-server:/biogrids-backup/

# Incremental backup (daily)
rsync -av --link-dest=../previous-backup /data/biogrids/ backup-server:/biogrids-backup/$(date +%Y-%m-%d)/

Migration Procedures

Migrating Between Systems

# On new system, restore configuration
biogrids-cli reactivate biogrids-installation-config.json

# Sync software from backup
rsync -av backup-server:/biogrids-backup/latest/ /data/biogrids/

# Rebuild environment
biogrids-cli rebuild

Legacy System Migration

For systems migrating from older BioGrids installations:

# Migrate from legacy installation
biogrids-cli migrate-admin

# Verify migration
biogrids-cli verify

# Update to current versions
biogrids-cli admin --all-versions

Best Practices

Recommended Configurations

Production Environment

# Complete installation with monitoring
biogrids-cli crontab --all-versions --linux --darwin -j 8

# Set up log rotation
sudo tee /etc/logrotate.d/biogrids << EOF
/home/biogrids/.biogrids/logs/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
}
EOF

Development Environment

# Selective installation for development
biogrids-cli install -c "Development Tools" -c "Programming Languages"

# Manual update schedule
biogrids-cli crontab --manual-updates

Monitoring and Alerting

Health Monitoring Script

#!/bin/bash
# biogrids-health-check.sh

LOG_FILE="/var/log/biogrids-health.log"
DATE=$(date '+%Y-%m-%d %H:%M:%S')

# Check disk space
DISK_USAGE=$(df /data/biogrids | tail -1 | awk '{print $5}' | sed 's/%//')
if [ $DISK_USAGE -gt 90 ]; then
    echo "$DATE: WARNING - Disk usage at ${DISK_USAGE}%" >> $LOG_FILE
fi

# Check for failed updates
if biogrids-cli verify | grep -i error; then
    echo "$DATE: ERROR - BioGrids verification failed" >> $LOG_FILE
fi

# Check network connectivity
if ! biogrids-cli check-connection > /dev/null 2>&1; then
    echo "$DATE: ERROR - Network connectivity failed" >> $LOG_FILE
fi

Security Considerations

Access Control

Use dedicated service account for BioGrids management
Implement proper file permissions and group access
Regularly audit user access and permissions
Monitor installation logs for suspicious activity

Network Security

Ensure outbound HTTPS access to biogrids.org
Configure firewall rules for required ports
Use VPN or secure networks for administrative access
Monitor network traffic for anomalies

For additional support with site administration, contact help@biogrids.org with details about your installation environment and specific requirements.