IMS3000 network switch and Internet access – any concerns

[Steve Guttag]

I don't deny that they need internet...however the Management network (the booth LAN) is not that. There is no internet there...just booth equipment.

Yes, I could understand having a separate network for the catch servers (we deal with 4 or 5 of them, depending on the site). Then firewall that off from the media network and only allow FTP transfers to specific addresses. I'm not there yet but I could understand that sort of restriction/system. There is the balance of functionality versus security.

[Bruce Cloutier]

You know, when there is a will there is a way. If a serious bad actor means to compromise your systems they will likely get it done. Meanwhile tho, it drives me nuts to watch random malicious bots attempting repeated logins from random IP addresses and often from the same dictionary of credentials. Logically you want to keep that noise off of your production networks. You can watch for it of course. I would be surprised if that appears on any of your networks.

If that leaks in, say while you open the network for some download, your default passwords represent a vulnerability. Sadly, early JNIOR have a secondary administrator account 'admin' (you can guess at the default password) and we have found that to be in a majority of the bot credential lists. Not so much the default credentials that we all use for those devices. People should disable that account.

That said, I put JNIOR on the open Internet leaving the factory configuration including the default accounts and password. Bots made successful logins but they all (so far) expect a Linux system and attempt to execute commands that JANOS does not recognize. You know, worms attempting to infect. So no problem. That is until one of those comes along targeting a JNIOR. I have not seen that as yet. As noted, we are not a "valued target".

Too much paranoia can be crippling. And too much neglect can be embarrassing. All you need is just enough care and vigilance. Couple that with backups and a recovery plan. It is a real shame that we have gotten ourselves to the point where this is even a thing.

[James Gardiner]

@Marcel — in practice, all content delivery networks require sites to install a dedicated internet connection. As a result, it’s not uncommon to see three separate dedicated internet services terminating in the TMS rack, each feeding a different delivery appliance, in addition to the front-of-house internet connection.

That means a site may be paying for four separate internet links.

Given the current pressure on the exhibition industry, this level of duplication is difficult to justify. For smaller independent cinemas in particular, it represents a material and unnecessary cost burden.

[Caleb Williams]

That means a site may be paying for four separate internet links.

When it was discovered that some 87% of transactions via point-of-sale came in via credit card, it only made sense to have a backup internet solution to prevent "cash-only" periods driving away customers. We've been operating that way for a year and with amount of outages we've had with a primary ISP, the secondary has paid for itself.

All of our locations have been upgraded to DCDC's broadband service, although we do not pay for that connection directly. I have seen a little bit of the system those servers run. As long as the rest of Deluxe's infrastructure is reasonably sound, I believe it would be difficult to infiltrate the on-site server without having physical access.

[Marcel Birgelen]

@Marcel — in practice, all content delivery networks require sites to install a dedicated internet connection. As a result, it’s not uncommon to see three separate dedicated internet services terminating in the TMS rack, each feeding a different delivery appliance, in addition to the front-of-house internet connection.

That means a site may be paying for four separate internet links.

Given the current pressure on the exhibition industry, this level of duplication is difficult to justify. For smaller independent cinemas in particular, it represents a material and unnecessary cost burden.

It happens, but it seems to be more prevalent in other countries than over here. If a site has properly managed their network, it's not uncommon to have some kind of Internet backup, but not a separate connection for every distribution service. If you have a somewhat decent firewall, you can also cap those distribution services to a certain maximum, especially if you've put them in a certain zone. Gofilex, for example, requires a 20 MBit/s minimum commitment. If you have GigE or more, it's really not an issue to have this traffic on your primary connection.

Also, we're quite lucky over here that you can get GigE or multi-GigE Internet speeds at very reasonable prices nowadays, even for businesses, at least if you can live without a 24/7 99.9%+ SLA. If you really need this kind of availability, I'd rather go for a multi-vendor backup solution.

It's actually more common to have a separate internet connection for public services like public hotspots, if they're offered at all. Public hotspots have the tendency to attract DDoSes.

One solution we apply is to use the existing Internet connection, but to put the public hotspot traffic into a VPN or VPN-like tunnel, proxying it out over a completely different external IP behind some DDoS filters. Meanwhile, applying QoS on the VPN tunnel. That way, anything nefarious happening on the public hotspot will not interfere with normal business operations. It also avoids the site getting blacklisted by the ISP, due to unwanted traffic on their Internet connection.

[Steve Guttag]

...

That said, I put JNIOR on the open Internet leaving the factory configuration including the default accounts and password. Bots made successful logins but they all (so far) expect a Linux system and attempt to execute commands that JANOS does not recognize. You know, worms attempting to infect. So no problem. That is until one of those comes along targeting a JNIOR. I have not seen that as yet. As noted, we are not a "valued target".

I'm wondering [always dangerous], could JNIOR (or some other device) deliberately do what you have done...have an easy to crack (know list of bot user/pw) there to just tally up IPs that are trying to infiltrate and start to firewall those IPs off until the attacks stop?

Furthermore, why can't countries, that love to regulate things, start to penalize ISPs that don't block such accounts (hit them where they get onto the net and most every point between). They're getting on the internet, somewhere.

[Marco Giustini]

I'd imagine most of that malware is coming from people completely unaware that their machine is being used to spread viruses and malware so it might be difficult to filter out. I believe it's why Microsoft got aggressive with updates from W10 on. Way too many people disabled everything with Windows 7 because it was inconvenient.

[Steve Guttag]

With Microsoft, I think it is also nefarious (feed their advertising)...they also are trying to stop local accounts...force you into one-drive...and so forth. Microsoft is almost as bad as malware and sometimes worse.

I still block Microsoft updates except when I want them. Since Microsoft is done with Win10, there is no need for those computers to call out to the mothership anymore.

I have no doubt that locating the source of malware is a daunting task, but the sooner ISPs start, the more they will be able to stop before they reach their targets. If there were financial incentive (large penalties) for facilitating such activity would also cut it way down.

[Bruce Cloutier]

Let's not run down the Microsoft rabbit hole and bash them. EVERYONE else is doing a fine job at that. MS has a captured audience and they are aggressively leveraging that in the name of greed. The other options each have their faults. The choice should be in favor of the lesser of the evils. Enough said.

I'm wondering [always dangerous], could JNIOR (or some other device) deliberately do what you have done...have an easy to crack (know list of bot user/pw) there to just tally up IPs that are trying to infiltrate and start to firewall those IPs off until the attacks stop?

I have two JNIORs directly on the Internet. Yes, we pay for two IP addresses for those.

As I mentioned, in factory configuration, they survive but were very frequently hit upon. The attacks fail due to unfamiliarity with JANOS. The network traffic, as you watch it, pisses you off within minutes. The situation improves if you set secure passwords, limit active accounts, and maybe close ports that you do not intend to use. Those two IP addresses we have had for years and were previously unused. You would think they would be very quiet. The first connection attempts occurred within minutes of initial connection.

You all know that I have authored JANOS and so have control at all levels. As an initial level of defense I leveraged a technique that email servers have used for years, Greylisting. However in this case JANOS can delay any initial TCPIP connection attempt. The initial SYN packet is ignored (as if there is no device there). The connection MUST be retried according to the standards. Not too soon and not too late. If retried properly the connection is allowed to proceed. This, after significant study, proves to thwart over 90% of malicious activity. The almost constant nefarious login attempts stop and such things drop to just a couple of dozen in a day! Normal communications are not impacted. The cybersecurity industry doesn't want to acknowledge this approach.

It is not a new thing. It can be considered to be a form of port knocking. But port knocking is positioned as a technique wherein both client and server have to knock correctly. This reluctant-SYN approach is the simplest aspect of that and it does not require anything of the client except for it to use a stack written to standards. Bots (and even chatGPT) do not do standards. This renders the existing population of worms mute. Google and the others do manage to crawl those units successfully. You will find links to these JNIORs in Google searches for topics in the users manual, for instance.

On top of that JANOS has the ability to blacklist IP addresses and IP address ranges. Failed login attempts, TLS negotiations, and website requests are logged. Both JNIORs are running Blacklister.jar which scans those logs and adds IP addresses to the blacklist. Those have just under 9,000 addresses blocked a the moment. The JNIOR is invisible to those blacklisted IP addresses. The JNIOR even detects probes. Those are the tricks used to find devices. Those are blacklisted and therefore cannot make use of what they learn and they can't ever detect us again.

All of this does wonders to thwart DoS attacks as well.

But you don't have to pay for an external IP address to have these issues. There are a lot of cases where port forwarding is used to pass communications through to a JNIOR. So all of the noise gets through onto the internal network with those forwarded ports.

Anyway, we have JNIORs accessible through the Internet controlling things like driveway gates and whatnot. We hope they use some kind of VPN. That said, they aren't doing any of the above. They probably don't even know that we can do any of that.

[Caleb Williams]

Coming from a Unix-ish background, JANOS is only barely familiar. Because the Jnior platform fits into the larger umbrella of SCADA-like devices, it's only a matter of time before bad-actors put JANOS in their sites. Good to hear you guys are taking security seriously from the ground up.