We Watched a Protected Web Server for One Night. The Internet Is Worse Than You Think.

What happens to an ordinary web server after you connect it to the public Internet—and why hosting companies, system administrators, web developers, and website owners need to start treating automated attacks as the permanent background radiation of the modern Internet.

I have been in the computer business since the Internet was young.

I remember when connecting computers together still felt inherently optimistic.

The Internet was going to democratize information. It was going to let small companies compete with large ones. It would connect people, businesses, universities, governments and communities in ways that had never been possible before.

It did all of those things. But somewhere along the way, we built something else too. We built the world's largest automated hostile environment.

Today, put a server on the public Internet and give it an IP address.

• You don't have to advertise it.
• You don't have to submit the site to Google.
• You don't have to have meaningful traffic.
• You don't even have to put anything particularly interesting on it.

Just make it reachable.

They will find it.

And when they do, they won't politely knock on the front door.

• They will try the windows.
• They will look under the doormat.
• They will check whether you forgot to lock the basement.
• They will try keys stolen from somebody else's house.
• They will ask your server for files no legitimate visitor should ever request.
• They will look for old applications you may not even have installed.
• They will probe your mail server, FTP server, SSH server, CMS, control panel and web applications.
• They will look for configuration files, environment files, Git repositories, credentials, backup files, forgotten administration interfaces and vulnerable plugins.

And most of them will do it automatically.

Twenty-four hours a day. Seven days a week.

This is not theoretical.

We recently spent a night examining the security telemetry from one of our production hosting servers while developing and testing Aegis Defender Pro, our server security and attack-correlation system. What we saw bothered me enough that I think people outside the security industry need to see it too.

Because here's the important part:

This wasn't an unprotected server. Quite the opposite. It was already heavily defended. And the Internet was still hammering on the doors.

This Wasn't a Honeypot

Security researchers sometimes publish fascinating reports from honeypots—systems deliberately exposed or made vulnerable so researchers can observe what attackers do. That's valuable research. But this wasn't that. This was a real production server.

• It had a firewall.
• It had CSF.
• It had ModSecurity.
• It had additional security tooling.
• It had geographic intelligence.
• It had IP reputation intelligence.
• It had automated attack detection.
• It had years of accumulated firewall blocks.
• It had entire hostile networks already excluded.

And it was running Aegis Defender Pro, which we have been developing specifically because we became dissatisfied with the idea that server security should consist of dozens of independent tools each shouting about its own little piece of the problem. Aegis watches multiple security channels and tries to answer a more important question:

What is actually happening to this server?

During our overnight examination, we saw:

• Probes for sensitive configuration data.
• Directory traversal attempts.
• Remote-file-inclusion-style behavior.
• Requests looking for things such as .env, .git and .aws configuration data.
• WordPress reconnaissance.
• Mail authentication attacks.
• Attackers coming from networks already associated with abusive behavior.
• Sources with extremely high AbuseIPDB confidence ratings.
• Attacks from individual addresses already covered by firewall CIDRs because related addresses had attacked previously.

We watched one foreign mail source arrive with 27 existing AbuseIPDB reports and a 91% abuse-confidence score. Our system classified it and blocked the source.

And that was just another event in the log. That's the part people need to understand. For the server, this isn't extraordinary. It's Tuesday.

---

“Why Would Anyone Hack My Website?”

I have heard variations of this question for decades:

“Why would anybody want to hack my little website?”

It's one of the most dangerous misconceptions in small-business cybersecurity.

The answer is:

They may not want to hack your website.

• They don't know you.
• They don't care about your company.
• They don't know what you sell.
• They may never have seen your homepage.
• A tremendous amount of hostile Internet activity is automated.

The attacker doesn't necessarily begin with:

“I want to compromise Bob's Plumbing in Roanoke Virginia.”

The process can be closer to:

• Find Internet-connected machines.
• Find machines that answer.
• Determine what they're running.
• Test known weaknesses.
• Look for exposed secrets.
• Try credentials.
• Find something exploitable.
• Compromise it.
• Move on.

At scale.

Your website isn't necessarily a target because someone selected you. It's a target because it exists. That distinction changes how you should think about security.

Imagine living in a world where someone walked past every house in America every night and tried every doorknob.

Nobody would say:

“Why would a burglar target my house?”

They aren't necessarily targeting your house. They're looking for an unlocked house. The Internet made that kind of reconnaissance extraordinarily cheap.

A human doesn't have to manually inspect every server.

• Software can do it.
• Bots can do it.
• Distributed networks of compromised machines can do it.
• Cloud servers can do it.
• Compromised hosting accounts can do it.

One attacker can examine enormous numbers of systems looking for one mistake. And defenders have to get the equation right every time. The attacker only has to get lucky once.

---

The Attacking IP Address Is Often the Least Interesting Part

This has been one of the biggest lessons we've learned while developing Aegis.

Traditional security logs tend to encourage a very narrow way of thinking.

You see:

185.x.x.x attacked the server.

Okay.

Block it.

Problem solved.

Except sometimes it isn't.

• Another address appears.
• Then another.
• Then another.
• Perhaps they're in the same subnet.
• Perhaps they're in the same ASN.
• Perhaps they're associated with the same provider.
• Perhaps they're requesting the same unusual resources.
• Perhaps they're hitting multiple websites on the server with nearly identical attack patterns.

Perhaps twenty seemingly independent IP addresses are actually pieces of one campaign.

That leads to a much more interesting security question.

Instead of asking:

Did 185.x.x.x attack me?

We should sometimes be asking:

Are these 37 IP addresses actually one attacker—or one coordinated attack campaign?

That's a fundamentally different way to look at server security. It is also why simply feeding individual IP addresses into a firewall isn't enough anymore.

The Whack-a-Mole Problem

Suppose an attacker controls one machine.

It attacks you.

You block it.

Done.

That's the easy case.

Now suppose an attacker can use hundreds or thousands of addresses.

• Address number one attacks. You block it.
• Address number two attacks. You block it.
• Address number three attacks. You block it.
• Address number four appears from the same infrastructure.

Congratulations.

You've invented an extremely expensive game of Whack-a-Mole.

And the attacker owns the mole factory.

This is why we've become increasingly interested in network-level correlation.

If enough trustworthy evidence tells us that multiple malicious addresses belong to the same network, the important security decision may eventually stop being:

Should we block this IP?

It becomes:

Should this network be allowed to talk to us at all?

That decision has to be made carefully.

You cannot simply start banning huge chunks of the Internet.

Cloud providers, CDNs, mobile carriers, universities, corporate networks and ISPs can contain both legitimate users and malicious systems.

Blocking an entire Cloudflare or Akamai network because one request looked nasty would be reckless.

Security isn't about being maximally aggressive.

Good security is about being maximally accurate.

That's considerably harder.

Reputation Helps. Your Own Evidence Matters More.

External reputation systems such as AbuseIPDB are extremely useful.

If an address knocks on your door and dozens of independent systems have already reported it for abusive activity, that's meaningful intelligence.

But there's a philosophical trap here. Imagine an attacker that nobody has reported yet. Does that mean it isn't attacking you? Of course not. Somebody has to be first.

That's why we believe a mature security system needs to distinguish between external intelligence and local evidence.

Those are different things.

If an IP has no AbuseIPDB history but spends ten minutes attempting authentication against several accounts across multiple services, we shouldn't necessarily wait for the rest of the Internet to tell us what we already know.

Reputation should strengthen evidence.

It should not replace observation.

A Firewall Is Necessary. It Is Not Enough.

This is another area where website owners can develop a false sense of security.

“My server has a firewall.”

Good. It absolutely should. But a firewall mostly answers questions about connections.  t doesn't inherently understand the semantic meaning of everything happening inside those connections. A web request for:

/.env

isn't interesting to a conventional packet firewall merely because of the characters in the URL.

To the firewall, it may simply be legitimate TCP traffic to port 443.

• The web server knows more.
• The WAF knows more.
• The application knows more.
• Authentication logs know more.
• Mail logs know more.

The challenge is getting those layers to communicate.

We need to stop thinking of server security as a collection of independent guards who aren't allowed to speak to each other.

Imagine protecting a building that way.

• The front-desk guard sees someone behaving suspiciously.
• The parking-garage guard sees the same person checking car doors.
• The loading-dock guard catches him trying an employee badge.

But none of them can communicate. Each incident individually might be ambiguous.

Together? It's obvious.

Computer security frequently has exactly this problem.

---

ModSecurity Is Excellent—If You're Actually Using It Properly

ModSecurity and modern WAF rulesets can stop an enormous amount of garbage.

But merely having ModSecurity installed isn't the same thing as having an effective web application firewall.

• Rules have to exist.
• They have to be appropriate.
• They have to be updated.
• They have to be monitored.
• False positives have to be managed.
• Logs have to be reviewed.

And most importantly: Something needs to happen when the WAF detects an attacker.

A security product generating a log entry is not necessarily the same thing as a security product protecting you. A burglar alarm that faithfully writes:

BURGLAR DETECTED

to /var/log/house.log but doesn't wake anybody up is technically an excellent detection system.

It's just not a particularly useful burglar alarm. That distinction has heavily influenced how we've built Aegis. Detection matters.

Enforcement matters too.

The Requests That Should Scare You

Some web requests are ambiguous.

Others really aren't.

• A visitor requesting your homepage? Normal.
• A search engine requesting robots.txt? Normal.
• A user mistyping a password? Normal.

Someone asking your web server for things such as:

• .env
• .git/config
• AWS credentials
• SSH keys
• System files
• Password files
• Arbitrary local paths
• Remote URLs intended for inclusion
• Forgotten database administration tools
• Known vulnerable application endpoints

That's different.

Legitimate customers do not browse your website by requesting your environment file. Google doesn't need your database password. A customer looking for your business hours does  not need /etc/passwd. At some point, excessive caution becomes absurd.

There are behaviors where the request itself is evidence of hostile intent. One of the things we've been refining in Aegis is identifying those conclusive signals. Because treating every security event identically makes about as much sense as treating someone walking past your house and someone drilling the lock on your back door as equivalent activities.

They're not.

---

Then There Is Email

Web attacks get most of the attention. Mail servers deserve some of it. We continuously see authentication attempts against IMAP, SMTP and other mail services.

Again, one failed password isn't necessarily an attack.

• People mistype passwords.
• Phones retain old credentials.
• Outlook does Outlook things.
• Users are users.

A good security system has to tolerate ordinary human mistakes.

But context changes everything.

• One failed login from a normal location? Probably nothing.
• Repeated failures? More interesting.
• Attempts against multiple accounts? More interesting.
• Attempts across multiple hosted domains? Very interesting.

An address in a foreign network with a substantial abuse reputation attempting authentication against an account it has never legitimately accessed?

Now we're accumulating evidence. This is why correlation matters. A security system shouldn't merely count failures.

It should understand behavior.

Geography Is Useful—But Geography Isn't Identity

We've also learned to be careful with country-based assumptions. An IP geolocated to the United States does not mean a human being sitting in the United States is operating it.

• Servers can be compromised.
• Residential systems can be compromised.
• VPNs exist.
• Proxies exist.
• Cloud infrastructure exists.
• Botnets exist.

Attackers deliberately route traffic through infrastructure in countries that defenders are less likely to block. Country information is useful.

It isn't proof of identity.

The same applies to providers.

• A Google Cloud IP isn't inherently Google.
• An AWS IP isn't inherently Amazon.
• It may simply be a virtual machine rented—or compromised—by somebody else.

Infrastructure ownership and attacker identity are not the same thing.

That sounds obvious when stated plainly. Security systems still get this wrong all the time.

“But We Already Blocked That IP.”

We saw something else interesting in our testing. Aegis would occasionally record hostile activity from an address and report that the source was already covered by the firewall. 

At first glance that sounds impossible. If it's blocked, how did we see it?

There are several legitimate explanations.

• The attack may have been logged just before the block became active.
• The log processor may be reading an event after firewall enforcement occurred.
• A connection may already have been in flight.
• The address may be covered by a CIDR promoted after the original event.
• A reverse proxy or CDN may mean the IP recorded at the application layer isn't the TCP source address visible to the firewall.

This illustrates another important point:

Security telemetry is a timeline, not a photograph.

• Detection time.
• Event time.
• Enforcement time.
• Firewall reload time.
• Application processing time.
• Correlation time.

Treat all of those as identical and you'll draw the wrong conclusions.

---

The Server We Watched Was Already Quiet

This may be the most disturbing part.

The production system we examined has been protected for a long time. 

• We have accumulated blocks.
• We have blocked individual attackers.
• We have blocked hostile CIDRs.
• We have identified networks repeatedly associated with attacks.

Over time, we've observed something that makes intuitive sense:

Some attackers eventually stop coming back.

If a bot repeatedly receives nothing useful from a network, there is little reason for its operator to waste resources indefinitely.

This means the hostile activity we see today may actually be the reduced version.

Think about that.

We were looking at the attacks that still reach a machine after years of active defense.

• What does a fresh server look like?
• What does a newly provisioned VPS look like?
• What happens to a machine with SSH exposed, no intelligent firewall policy, a weak WAF, an outdated CMS and nobody actually reading the logs?

I think many people would be horrified if they could simply sit beside the server and watch.

---

Silence Does Not Mean Nobody Tried

This is perhaps the biggest psychological problem in cybersecurity.

Successful defense is invisible.

• If someone compromises your server, everybody notices.
• If someone encrypts your files, everybody notices.
• If customer information gets stolen, everybody notices.
• If Google marks your site malicious, everybody notices.
• If your IP gets blacklisted because a compromised account sent 600,000 spam messages, everybody notices.

But suppose last night:

• 400 reconnaissance attempts failed.
• 27 authentication attacks failed.
• 18 malicious networks were already blocked.
• Three vulnerable-file probes were stopped.
• Two credential attacks were correlated.
• A malicious IP was blocked before its second request.

And nothing bad happened.

What does the website owner see the next morning?

Nothing.

• The website works.
• Email works.
• Orders arrived.
• The phone rings.
• Business continues.

Security succeeded.

And because security succeeded, it can look like nothing was happening.

That creates a dangerous feedback loop.

“Why are we spending money on security? Nothing ever happens.”

Exactly.

That's the desired outcome.

---

What Happens When We Don't Take This Seriously?

This is where I think the technology industry needs to have a much less comfortable conversation. A compromised website isn't always the attacker's final objective.

It's infrastructure.

A compromised web server can become another machine used to attack somebody else.

• It can host phishing pages.
• It can distribute malware.
• It can send spam.
• It can participate in credential attacks.
• It can scan other servers.
• It can become part of a botnet.
• It can become a command-and-control node.
• It can be used as a proxy.
• It can host stolen data.
• It can participate in denial-of-service attacks.
• It can help attackers hide where they actually are.

That innocent little server somebody didn't bother securing can become somebody else's attacker tomorrow.

That's the endgame that worries me.

Every poorly maintained server isn't merely a risk to its owner. It can become additional capacity for the hostile ecosystem.

• One compromised machine attacks ten more.
• Some percentage of those become compromised.
• Those machines attack others.
• The attacker becomes increasingly abstracted from the infrastructure actually generating the traffic.

We end up fighting servers owned by innocent companies, cloud instances purchased with stolen credentials, compromised residential systems and networks whose legitimate owners have no idea what's happening.

The Internet starts attacking itself.

And increasingly, that's what the Internet looks like.

---

Ransomware Doesn't Materialize Out of Thin Air

People often experience ransomware as an event. One day everything works. The next day there's a ransom demand. But the final payload is only one stage.

Something came before it.

• Reconnaissance.
• Initial access.
• Credential compromise.
• Exploitation.
• Persistence.
• Command and control.
• Lateral movement.
• Privilege escalation.
• Data discovery.
• Exfiltration.
• Then perhaps encryption.

By the time the ransom note appears, the security failure may be days, weeks or months old.

That's why watching the edges matters.

• The weird login attempt matters.
• The strange request matters.
• The unknown process reaching an unfamiliar network matters.
• The file that unexpectedly changed matters.
• The authentication attempt against three accounts matters.

Individually, many of these things look small.

Collectively they may tell a story.

The Future of Defense Is Correlation

I believe this is where server security has to go.

Not more logs.

We already have plenty of logs.

We need understanding.

• A firewall knows about connections.
• A WAF knows about HTTP attacks.
• The mail server knows about authentication.
• SSH knows about SSH authentication.
• The CMS knows about CMS logins.
• File-integrity monitoring knows what changed.
• External reputation services know what other systems have observed.
• Geolocation knows where an address appears to originate.
• ASN intelligence knows who operates the network.
• Provider classification tells us whether an address belongs to residential broadband, hosting infrastructure, cloud computing or a CDN.

Each knows part of the story. The opportunity is to put the story together. That's what we're trying to do with Aegis Defender Pro. And that's why one of the features I'm most excited about is something deceptively simple:

Campaigns.

Instead of:

• IP A attacked.
• IP B attacked.
• IP C attacked.
• IP D attacked.

We want the system to be able to say:

These 42 addresses appeared over three hours.

They came from related infrastructure.

They targeted 11 hosted websites.

They requested the same vulnerable resources.

They attempted the same exploit family.

31 were previously reported for abuse.

17 were already covered by firewall policy.

8 individual addresses were blocked.

3 networks crossed the escalation threshold.

This appears to be one campaign.

Now a system administrator can understand what's actually happening.

That's much more valuable than line 847,293 in an access log.

---

Automation Needs Guardrails

There's an obvious danger here.

If automated defense becomes too aggressive, it becomes its own denial-of-service attack.

Imagine software automatically blocking Google, Cloudflare, Microsoft or an ISP serving half your customers because it misunderstood one event.

That's why automated security needs brakes as much as it needs an accelerator.

We've spent a considerable amount of development effort on questions like:

• Is this IP protected?
• Is it local?
• Is it explicitly allowed?
• Is this a CDN?
• Is this cloud infrastructure?
• Is the evidence strong enough for an exact-IP block but not a CIDR?
• Is there independent reputation evidence?
• Have multiple addresses participated?
• Are we seeing one site or many?
• Was this attack conclusive?
• Is this event fresh?
• Has this source already been blocked?
• Would this CIDR encompass an unreasonable number of unrelated addresses?

Automation without context is dangerous.

But refusing to automate anything because automation can be dangerous isn't an acceptable answer either.

Machines are attacking at machine speed.

Human beings cannot manually review every event.

The defender needs machines too.

---

Hosting Companies Need to Take This Seriously

This is where I'll probably irritate some people.

Fine.

Hosting companies have a responsibility here. Web developers have a responsibility here. Server administrators have a responsibility here. And companies selling “managed hosting” definitely have a responsibility here.

• Putting a customer's website on an Internet-connected server and making sure Apache starts isn't management.
• Updating WordPress occasionally isn't a complete security strategy.
• Installing a firewall and forgetting about it isn't a security strategy.
• Installing ModSecurity with whatever happened to ship with the machine isn't necessarily a security strategy.
• Collecting logs nobody reads isn't a security strategy.
• Sending an email after the compromise isn't a security strategy.

We have to stop treating server security as an optional accessory.

If you manage infrastructure for other people's businesses, you are managing part of their security perimeter whether you asked for that responsibility or not.

• Their customers trust that server.
• Their employees trust it.
• Their email may run through it.
• Their credentials may touch it.
• Their customer data may live on it.
• Their reputation depends on it.

And if it gets compromised, machines belonging to completely unrelated people may eventually be attacked by it.

This isn't just an uptime problem.

Website Owners Need to Ask Better Questions

Website owners don't need to become system administrators.

But they should ask their providers some questions.

Not:

“Do you have security?”

Everyone will answer yes.

Ask:

• What happens when someone repeatedly attacks my site?
• Do you run a WAF?
• Is it actively maintained?
• Do you monitor authentication attacks?
• Do you correlate attacks across services?
• Do you automatically block confirmed malicious sources?
• Do you distinguish individual IPs from coordinated networks?
• Do you use external reputation intelligence?
• Do you protect against false positives when automating firewall decisions?
• Do you monitor file changes?
• Do you know when an attacker probes for configuration files?
• Do you know when the same source attacks several customers?
• Does anybody actually review this information?

And here's my favorite:

How would you know if somebody tried to compromise my website last night?

If the answer is vague, that's useful information.

We Need to Stop Measuring Security Only by Breaches

This is the conclusion I've reached after watching these systems for years. The security industry naturally studies successful attacks. We should. That's how we learn.

But that means the public mostly hears about security when defense has already failed.

• Ransomware.
• Data breaches.
• Stolen passwords.
• Defaced websites.
• Leaked databases.
• Millions of customer records.

Those are the spectacular failures.

There's another dataset almost nobody sees:

The attacks that didn't work.

That dataset may tell us even more about what the Internet has become.

Every night, all over the world:

• Servers are being tested.
• Passwords are being guessed.
• Vulnerabilities are being probed.
• Configuration files are being requested.
• Networks are being scanned.
• Old software is being searched for.
• Mail accounts are being enumerated.
• Credentials are being tried.

And most of it never becomes a news story.

• Some of it fails because the vulnerability isn't there.
• Some fails because software is patched.
• Some fails because a WAF recognizes it.
• Some fails because an IP has already been blocked.
• Some fails because a security system sees the pattern forming.
• And some succeeds.

That's the part that should bother everyone.

I Never Thought the Internet Would Become This

I've spent essentially my entire professional life around computers. I watched the Internet grow from something unusual into something civilization depends upon. And I still believe it is one of the greatest technologies humanity has ever created.

But I never imagined that simply connecting a machine to this network would eventually mean exposing it to a continuous automated population of scanners, exploit tools, credential thieves, botnets and compromised machines looking for the next place to get in.

Yet here we are.

We built an extraordinary global network. Then we connected nearly everything important to it.

• Banking.
• Hospitals.
• Government.
• Transportation.
• Commerce.
• Communications.
• Utilities.
• Businesses.
• Homes.
• Phones.
• Cameras.
• Factories.
• Schools.

And then we populated that same network with an enormous amount of automated malicious activity. That combination should make us uncomfortable. Because there is an endgame if we don't improve this.

As more poorly protected machines are compromised, attackers gain more infrastructure.

That infrastructure attacks more machines.

Compromised systems become proxies for new attacks.

Attribution becomes harder.

Automation improves.

Attack tools improve.

Artificial intelligence will inevitably improve both offensive and defensive automation.

The speed increases.

The scale increases.

And eventually the assumption that a public Internet connection is a fundamentally hostile environment stops sounding paranoid.

It becomes basic systems engineering.

Maybe we're already there.

Put a Server on the Internet and Watch

That's ultimately what prompted me to write this.

Not a research paper.

Not a vendor presentation.

Not a scary cybersecurity advertisement.

A log file.

• We watched a protected production server.
• We watched what came through the door.
• We watched Aegis identify attacks.
• We watched reputation intelligence confirm some of them.
• We watched exact addresses get blocked.
• We watched addresses already covered by previous network blocks appear again in the telemetry.
• We watched automated systems search for things no legitimate visitor should ever request.

And remember:

This was the protected server.

This was the machine where many of the repeat offenders and hostile networks had already been told, repeatedly, to go away.

Imagine the raw one.

• Fresh IP.
• Fresh VPS.
• Default configuration.
• No accumulated firewall intelligence.
• Poorly configured WAF—or none at all.
• Nobody watching the logs.
• Nobody correlating the authentication attempts.
• Nobody noticing that five IP addresses are actually behaving like one attacker.
• Nobody noticing the reconnaissance that came before the exploit.
• Nobody noticing until something stops working.

That's the server I worry about.

Because somewhere on the Internet, thousands upon thousands of those machines exist right now.

Some haven't been compromised.

Yet.

Some probably have been compromised and their owners don't know it.

And some of the machines attacking us tonight may be among them.

That's the vicious circle.

The computer attacking your server may not belong to a criminal.

It may belong to another victim.

---

The Goal Isn't to Win Every Fight

There's one final lesson our experience has taught us.

You don't necessarily need to defeat every attacker.

You need to become not worth their time.

• Recognize them.
• Block them.
• Recognize the network when the evidence supports it.
• Remember what you've learned.
• Share intelligence between detection systems.
• Make repeated attacks progressively less productive.
• Make scanning you expensive and unrewarding.
• Make the bot receive nothing useful.

Again.

Again.

Again.

Eventually, some of them move elsewhere.

That's not glamorous cybersecurity.

There is no movie scene.

No hacker furiously typing in a dark room.

No giant red ATTACK STOPPED animation.

There's just this:

Someone tried.

They got nowhere.

The system remembered them.

Business continued.

And tomorrow morning, nobody knew anything happened.

After watching what actually hits a public server at night, I have come to believe that may be one of the best definitions of successful cybersecurity there is.

Nothing happened.

But not because nobody tried.

Nothing happened because somebody was watching.