Linux nerd and consultant. Sci-fi, comedy, and podcast author. Former Katsucon president, former roller derby bouncer. http://punkwalrus.net

  • 0 Posts
  • 19 Comments
Joined 1 year ago
cake
Cake day: June 22nd, 2023

help-circle

  • Probably HR (or the NCS equivalent) never told the right people. I am not saying this is actually what happened, but a lot of IT bemoan the fact they are never told some rando employee was fired because HR neglects to inform them. Sometimes it takes months to discover, and even with a 90 day password/login lockout, some halfway decent admin could get around this by secretly building a back door, and using the messed up communication and politics between departments to hide this. Even in the 1990s, I saw people put in “time bombs” in their code that “if such and such is not updated in 6 months, run destructo-script A.”

    But imagine someone like Kandula Nagaraju here. Worked in QA, probably did a great jobs with some skills, but had the personality of swallowing broken glass. He was terminated in October 2022 due to “poor work performance,” which could mean anything. “Not a team player.” Or maybe he really was an idiot: I mean, a smart person would have a conniption, but get employed elsewhere and then slam his former company at parties. “Those NCS folks didn’t know what they had with me!” But this guy was probably someone with some anger management issues, probably a jerk, and possibly stupid. He might have had revenge fantasies, and set up a small virtual server posing as a backup code mirror. But outside the audits, it allowed ssh from the outside, and hid it through a knockd daemon. Or maybe only launched ssh at certain hours before shutting it down again. Silently working away in a sea of virtual servers with little to no updated documentation. He gets in, has internal access, and runs a script with admin credentials because they don’t rotate their AWS keys/secrets quickly enough. Or didn’t even know he was let go.

    After Kandula’s contract was terminated and he arrived back in India, he used his laptop to gain unauthorised access to the system using the administrator login credentials. He did so on six occasions between Jan 6 and Jan 17, 2023.

    That’s embarrassing to the company. Not only did he get in, but SIX TIMES after he was let go. he probably knew what order to run the delete commands (like, say, an aws “terminate-instances” cli command from a primary node), and did so one by one, probably during hours with the least amount of supervision, where the first few alerts would take hours to get someone in the monitoring chain to wake an admin. Given his last day was in November, and he got back in January, the admins probably thought their 90 access credential rotation was “good enough,” but he got in on his 80th day or whatever.

    I know this because I have had to do triage when a former contractor did this to a company I worked for. But instead of wiping out instances, he opened a new set of cloud accounts from the master account, put them in an unmonitored region (in this case, Asia), and spun up thousands of instances to run bitcoin mining. Only because AWS notified us of “unusual traffic” were we made aware at all, and this guy knew his shit and covered his tracks very well. He did it at a speed that could have only been automated. Thankfully, AWS did not charge us the seven figure amount that this activity amassed in just three days.


  • I remember hearing that some Hollywood contracts require that if you sign up for some studio, you must make X amount of films. Big stars get to chose those films to some degree, but once in a while, they have to do “a stinker” to end the contract as “X amount of films done, okay?” or something. Contractual Obligation and all. This film feels like a dumping ground of a lot of those contractual obligation hires from the trailer alone.


  • The thing is that for a majority of cases, this is all one needs to know about git for their job. Knowing git add, git -m commit “Change text”, git push, git branch, git checkout , is most of what a lone programmer does on their code.

    Where it gets complicated real fast is collaboration on the same branch. Merge conflicts, outdated pulls, “clever shortcuts,” hacks done by programmers who “kindof” know git at an advanced level, those who don’t understand “least surprise,” and those who cut and paste fixes from Stackexchange or ChatGPT. Plus who has admin access to “undo your changes” so all that work you did and pushed is erased and there’s no record of it anymore. And egos of programmers who refuse any changes you make for weird esoteric reasons. I had a programmer lead who rejected any and all code with comments “because I like clean code. If it’s not in the git log, it’s not a comment.” And his git comments were frustratingly vague and brief. “Fixed issue with ssl python libs,” or “Minor bugfixes.”


  • Not just LinkedIn profiles: there was a case out here near DC a while ago where a well known company leased out their function space for training meetings. Using a compromised company account, a set of scammers set up some fake recruitment profiles, leased out the meeting space for “software training,” and did some “mass hiring” where 30 individuals had their credentials scanned and duplicated. The effect was someone from the recruiting company was contacting you, you had a face-to-face where you got offered an in-person, you showed up to their offices, and got a “job offer pending a background check,” with a date of hire in official-looking emails. You sent in your SSN, copies of your passport and driver’s licence, and after a few weeks, they tell you to show up for orientation. Only, the day these people showed up, the company was confused and had never heard of you. The people you supposedly spoke to had never heard of you. And your identity was stolen, and huge loans and charges started showing up in your credit report.

    Yikes.


  • I had an issue with a UK supplier shipping to me in the US because UK mail is owned by DHL. Two shipments made it to the US, made it to New York, made it through customs and then… “Attempted Delivery, Kein Standort ausgewiesen, US.” Never seen again.

    DHL kept trying to convince me and the seller that “Your German address is refusing shipment.”

    “Last delivery hop was Flushing, NY.”

    “No, last delivery hop was in Kein Standort ausgewiesen. That company refused shipment, it went back to the UK.”

    “‘Kein Standort ausgewiesen’ is German for ‘No location reported,’ dumbass. Where is my package?”

    “… We don’t know. Germany somewhere. Call them.”

    Thankfully my vendor has refunded my money. Sadly, he no longer will ship to the US because this keeps happening to his customers. :(



  • “The simple act of coordinating human resource decisions with IT department actions, such as revoking account access for dismissed personnel, would significantly mitigate such risks.”

    HAHAHAHAH… ahh… yeah, like that’ll ever happen. How do I know my users have been deleted? When I find out by accident 90 days later. I have worked for several companies where HR doesn’t do shit for IT. I find out employees have been hired when they show up to my desk, asking for a login and laptop. I find out they were let go when the 90 day expiration report shows who got expired for not logging in for 90 days. “Jim’s admin password expired.” “Jim left in October.” I have worked for companies where the simplest of forms, a form generated by an email or a popup in some ticketing system only requires checking off a checkbox… nothing. Can’t be arsed.





  • A friend of mine had an expensive LED TV set get bad RAM about 10 years ago after a firmware fix. You could watch TV for about 2 hours before it went blank. Only official fix via the manufacturer was to disconnect it from power, wait until the rechargeable battery went down, then it was fine again for another 2 hours. It seems like it’s overheating, but it’s not. Something to do with a memory leak and video buffering. It was a known issue among tech enthusiasts, there was a homemade wiki on how to replace the shitty low end RAM with a $30 stick of laptop DIMM and it worked! He still has it, I think.


  • Being poor. In college in the 90s, my lead sysadmin couldn’t afford Minix for this system we had, so we tried to compile Linux on it. Three days later, we still failed, and gave up, but this was kernel 0.93 or something, so it had a ways to go. But I learned so much from that experience without paying for a university course or something.

    Years later, I bought a copy of Red Hat 6 at a Costco. Windows 95/98 was big, I didn’t know how to pirate it, so I went back to Linux and it worked great on my “franken-puters” cobbled together from spare parts dumpster diving. Steep learning curve back then, though. Then I brought it to my workplace, went from UNIX admin to Linux admin, and soon I preferred it to Windows. Been my daily driver for decades, now.

    Am I an evangel? A little, but I find that “right tool for right job” is a better approach. Linux is great for everything, BUT a comprehensive system like MS Office AND Active Directory simply does not exist in FOSS space yet; everything is cobbled together and a kludge still trying to catch up.

    Obsessed? Kinda. I just assembled some ansible scripts to roll my own distro. Why? To see if I could.


  • Punkie@lemmy.worldtoLinux@lemmy.ml*Permanently Deleted*
    link
    fedilink
    arrow-up
    27
    ·
    edit-2
    11 months ago

    I worked in a job with build scripts. Developers would list what they wanted in a drop-down menu on a website, with very few “fill in the blanks.” This would create a template, which was sanity-checked.

    One of the “fill in the blanks” was “home directory of user, if not default /home/username.” Some people filled it in, some didn’t. A lot of “users” might be apps with /home being “/opt/appname” “/var/www/html” or something. We checked to make sure that directory existed, if not, create, and set permissions. Easy peasy, all automated. Ran this lots of times.

    Then one day, the script failed. Borked the whole box. Sometimes the VM was corrupt, so delete VM and try again. Usually worked. But this time, the build kept failing. The box went down. Wasn’t even bootable. This happened several times with this one build. So we mounted the borked drive under a new VM and checked out the logs. Just like the dessert stage of Willy Wonka chewing gum, it always failed at the last stage: making /home directories.

    It would create them, then halt that it could not find bash. We looked for bash on the bad drive, and it was the usual /bin/bash shortcut to /usr/bin/bash and we were truly puzzled. I did a chroot to the drive and NOTHING worked. It just hung. That was the first clue.

    The second was looking through the build script (in bash, which we didn’t write) and checking the steps. Looked it the logs. Always died at creating some user named sapadm, the user for the HANA database. Eventually, I checked the configure file, and noticed it was the only user with the odd home directory “/usr/sap.” Then it hit me: the permissions.

    The script, thinking it was a home directory, did a chmod - R 755 for all directories and chmod - R 644 for all files! That meant, while creating home, it made everything under /usr not executable anymore! Holy shit, no wonder nothing worked! So we commented out that user in the config, ran the build again, and we were good! We created the sapadm by hand, and then later fixed the bug in the script.

    SANITIZE YOUR DATA. Or you might turn Violet Beauregarde into a blueberry.






  • Maybe not the same, but a knoppix CD was part of my toolkit for field work for many years. Stuff I did with it:

    • Retrieve or fix data from systems that could not boot.
    • Scan systems infected with boot viruses (clamscan), and wipe entire drives if necessary
    • Test various network issues: DHCP, DNS, tcpdump, and so on because Windows tools were pretty bad for a while
    • Bypass various Windows restrictions on user’s systems