Sad Servers:
Linux challenges
As a associate system administrator I worked on Redhat Linux servers, including user management, permissions, services, and performance monitoring Automated routine administrative tasks using Bash scripting and cron jobs, reducing manual effort by ~30% I am aws certified sysops administrator and Google Certified Cloud Engineer. Determined to transition my career into cloud architect /Cloud Support role
As System Administrators and DevOps Engineers, we often face situations where a combination of Linux commands, log analysis, and systematic troubleshooting helps us resolve issues quickly.
Recently, I worked through several hands-on Linux challenges that tested skills in text processing, log analysis, web server troubleshooting, and database administration. Here are the solutions and the thought process behind them.
- Santiago Challenge: Decoding a Hidden Secret
The Problem:
A spy named Alice has hidden a secret code across multiple text files.
The objective is to:
Count all occurrences of the word "Alice" across every .txt file.
Find the file where "Alice" appears exactly once.
Extract the number located immediately below that occurrence.
Combine both values into a single secret code and save it to a file.
How to Approach It:
This challenge demonstrates the power of Linux text-processing tools.
Step 1: Count All Occurrences
Search across all text files and count every matching line containing the word "Alice".
Step 2: Identify the Unique File
Use pattern matching to locate the file where the keyword appears only once.
Step 3: Extract the Hidden Number
Display the matching line and the following line, then isolate the numeric value.
Final One-Liner
echo \((grep Alice -h *.txt | wc -l)\)(grep -A1 Alice \((grep -c Alice *.txt | grep :1\) | cut -d: -f1) | grep -oEe "[0-9]+") > /home/admin/solution
Pattern matching
Command substitution
- Saskatoon Challenge: Finding the Most Active Client IP
The Problem:
A web server access log contains thousands of requests.
The task is to identify the IP address responsible for the highest number of requests and store it in a file.
How to Approach It
Log analysis is a critical skill for System Administrators and Site Reliability Engineers.
Step 1: Extract IP Addresses
Use regular expressions to isolate valid IP addresses from the log file.
Step 2: Count Occurrences
Group identical IPs and count how many times each appears.
Step 3: Identify the Top Requester
Sort by request count and select the highest value.
Final One-Liner
grep -oEe '[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}' access.log | sort | uniq -c | sort -n | tail -n1 | awk '{ print $2 }' > /home/admin/highestip.txt
Key Skills Demonstrated:
Log parsing
Regular expressions
sort
uniq
awk
- Tokyo Challenge: Resolving a Broken Apache Web Server
The Problem
A web page exists at:
/var/www/html/index.html
However:
curl localhost
returns:
403 Forbidden
and
curl 127.0.0.1:80
hangs indefinitely.
Troubleshooting Methodology
Rather than guessing, follow a structured approach.
Issue #1: Permission Denied
Reviewing Apache error logs reveals that the web server lacks permission to read the file.
Example:
Permission denied
The file permissions are overly restrictive:
-rw-------
Resolution:
Grant read access so Apache can serve the content.
chmod a+r /var/www/html/index.html
Issue #2: Port 80 Traffic Blocked
Inspecting firewall rules shows that incoming HTTP traffic is being dropped.
Example:
-A INPUT -p tcp --dport 80 -j DROP
Solution
Remove the blocking rule.
iptables -F
Final Fix
chmod a+r /var/www/html/index.html ; iptables -F
Key Skills Demonstrated
Apache troubleshooting
Log analysis
Linux permissions
Firewall administration
Network diagnostics
Manhattan Challenge: PostgreSQL Database Write Failure
The Problem
PostgreSQL is unable to write data to the database.
Applications attempting inserts or updates encounter failures.
Troubleshooting Methodology
A healthy database requires available disk space.
Step 1: Verify Storage Usage
Check filesystem utilization:
df -h
The output reveals:
100% Used
on the database storage partition.
Step 2: Locate Unnecessary Files
Inspect the data directory:
ls -lhA /opt/pgdata
Several large backup files are consuming available storage.
Example:
backup1.bk
backup2.bk
backup3.bk
Step 3: Reclaim Space
Remove obsolete backup files and restart PostgreSQL.
rm -f /opt/pgdata/*.bk
systemctl restart postgresql
Key Takeaways:
These challenges highlight an important lesson for every Linux Administrator, Cloud Engineer, and DevOps Professional:
✅ Start with observation before action.
✅ Use logs as your primary source of truth.
✅ Understand how Linux utilities can be chained together to solve complex problems.
✅ Always follow a systematic troubleshooting methodology instead of relying on trial and error.
The ability to quickly diagnose issues involving permissions, networking, storage, logs, and services is what separates experienced engineers from those who simply memorize commands.