Preface

This work manual, next to my life manual, is intended to help me and people dear to me get the most out of our precious time on this planet. I would highly appreciate if you would kept this document private and safe since I am writing it for myself only. You received a copy because you earned my trust and respect && I though you might find parts of it interesting and/or useful.

This document is evolving on daily basis which means your version is probably already outdated. Furthermore many of materials I acquire and produce are not and will not be listed here, therefore I encourage you to inquire me about parts you’re interested in, or ask for a new copy either in real life or by sending me email both to tom at triglex.com and kucar.tomislav at gmail.com

Notes to the Reader

During my formal education I studied economics & business for 4 years and after that software engineering for 3 years, hence most of this initial manual will be focused on that. As I progress in my career path I will update and rewrite when necessary.

Overview of manuals structure
  • Development: set of sorted resources and tasks to become proficient at a given domain

  • Business: set of sorted generic resources on business and startups

  • Notebooks: my digital notebooks, mostly notes and cheatsheets

Additionally, for physical products manufacturing resources and safety handbook see workshop manual. You can find the Triglex internal handbook under resources on Triglex Web Services dashboard.

Acknowledgments

Thanks to my primary school staff for forgiving me so many times, if it wasn’t for you I would never finish my primary education.

Thanks to Kristina Bilota for helping me grow up as a person and an artist.

Thanks to Dijana Stolfa, Adrijana Zic and other highschool teachers for making and teaching a high quality, not only bearable but enjoyable economy program that propelled me towards business.

Thanks to Daniel Bele for his incredible passion in teaching, he helped me not completely hate university. I’d especially like to thank Daniel for not burning me alive for my half-assed exams and homeworks, and for unknowingly motivating me to progress without being scared of not leaning on people around me. I’ll cherish his stories and anecdotes for the rest of my life.

Thanks to Mario Fabijanić for letting me pass my database course, twice. I actually learned them after that. Thanks to Ratka Jurkovic and Nebojsa Stanic for motivation and applicable advice.

Thanks to Dora Dasovic and Karlo Kegljevic for being my partners in work, crime and life. You helped me become a better person.

And finally thanks to my parents who will never read this. It took me 18 years to realize how much you have done for me and how alike we are. I love you.

Development

Design, UX and UI

Why learn UI design?
  • UX Designers. You can present designs as beautiful mockups that your coworkers will rally around. You can work with interfaces from concept to pixel-perfection. Your portfolio stands light-years ahead of your peers.

  • Developers. You can cover for design on your team. You can spiff up your side-project’s interface, instead of finding/hiring a designer. You can translate designs to code with minimal hassle, because you understand the aesthetic underpinnings.

  • PMs. You can create amazing mockups for pitching new features and flows. You can work alongside your designers, and provide spot-on feedback for excellent product.

  • Entrepreneurs. Like it or not, a good presentation for your ideas matters. Your marketing site, your app – even your pitch deck. Eventually, you’ll be in the big leagues and can hire out – but even then, a solid foundation in design will help you communicate and lead.

  • Print Designers. More and more design work shifts to digital every day. You’ve got to learn the ropes, figure out this whole "responsive" business, and practically re-build your portfolio to keep working in the field you love.

Books

  • The Design of Everyday Things: Revised and Expanded Edition

  • The Non-Designer’s Design Book

  • User Interface Design for Programmers by Joel Spolsky (three days)

  • About Face: The Essentials of Interaction Design

  • Don’t Make Me Think, Revisited: A Common Sense Approach to Web Usability

Resources

Practice

Practicing and learning by yourself is the only thing that can make you a better designer

Roadmap (iterate)
  1. Familiarize yourself with UI principles.

    • Color & Contrast

    • Composition & Balance

    • Typography

    • Consistency

  2. Learn the creative UX process.

    • Discover - researching, getting inspired, and gathering ideas, get to now users, their wants and needs

    • Define - define an idea extracted from the Discover phase. From this, a clear creative brief is created

    • Develop - This is where solutions or concepts are created, prototyped, tested and iterated. This process of trial and error helps designers to improve and refine their ideas

    • Delivery - The final phase is the delivery stage, where the final project is finalised, produced and launched

  3. Design fake projects/case studies

  4. Read design articles

  5. Look at inspiration artists/pages/etc

  6. Mentor and get mentored


Frontend Web Dev

frontend webdev paycheck in 2018
frontend2020

Web fundamentals

Basics

Projects
  • google.com homepage clone (no frameworks, no dev tools)

  • CSS garden (download html file and make style.css for it)

  • pick and recreate complex website (eg admin dashboard template)

  • build a CSS framework and demo page for it

Coding

Projects
  • build a calculator or pomodoro timer

  • build github api client using your CSS framework and jquery (optional)

Advanced

projects

Extra


Backend Web Dev

Backend webdev paycheck in 2018
backend2020

Python

  • Hitch hikers guide to python

  • Official documentation

Intro
  • Automate the Boring Stuff with Python 2nd ed

  • Byte of python

Medium
  • Dive in python

  • Learning Python 5th Edition (A comprehensive language foundations tutorial 1640pg, [3.3, 2.7])

Advanced
  • Programming python (hard, A full-scale applications programming tutorial 1630 pages [3.2])

  • Fluent python (A great intermediate Python book for those looking for the next step.)

  • Effective python (Also a great intermediate Python book for those looking for the next step.)

Other
  • How to make mistakes in python

  • Python Tricks: A Buffet of Awesome Python Features

  • Python Testing with pytest

  • Obey the Testing Goat!

SQL

  • Practical SQL: A Beginner’s Guide to Storytelling with Data

  • SQL Antipatterns: Avoiding the Pitfalls of Database Programming

Flask

Flask web development 2nd ed is much shorter, intermediate level, and more focused on flask on which it goes into detail compared to the flask mega tutorial which is bigger and includes many topics (such as full text search, ajax, javascript, localization and internationalization) and its intended for beginners.

Archiehcture

Software engineering

old backend
Old roadmap

DevOps / SRE

Devops salary in 2018
devops2020
Prerequisites
  • OS concepts (covered by comp sci)

  • backend engineering roadmap

Books

Software engineering

Programming languages

C
  • K&R C

  • Expert C Programming: Deep C Secrets

C++
References
Intro (no previous programming experience)
  • Beginning C++ Through Game Programming

  • Programming Principles and Practice Using C++

  • C++ Primer (covers just about everything in the language C++11)

Intro (with previous programming experience)
  • C++ Crash Course (C++17)

  • A Tour of C++ 2nd ed (tutorial overview of all of standard C++17)

Intermediate
  • Effective Modern C++ (best practices, C++11/14)

  • C++ Concurrency In Action 2nd ed (C++14/17)

C#/.NET
  • The yellow book

  • C# in Depth

  • C# 7.0 in a Nutshell: The Definitive Reference

Clojure
  • Clojure for the Brave and True

  • Clojure in Action, Amit Rathore, 2011.

  • Clojure for Machine Learning by Akhil Wali

Haskell
  • Learn You a Haskell

  • Beginning Haskell: A Project-Based Approach

Java
  • Introduction to Programming Using Java by David Eck

  • Java for impatient

  • Effective Java

Lua
  • Programming In Lua (that’s it, just read lots of source code now.)


Programming

Test-Driven Development

  • Test Driven Development: By Example

Design Patterns

  • Head First Design Patterns by Eric Freeman (1-2 weeks) (intro)

  • Pattern-Oriented Software Architecture v1, v2

  • Design Patterns: Elements of Reusable Object-Oriented Software (use as ref for POSA)

  • Smalltalk Best Practice Patterns

Architecture and system design

  • Patterns, Principles, and Practices of Domain-Driven Design

  • The Architecture of Open Source Applications

  • Domain-Driven Design: Tackling Complexity in the Heart of Software

  • Patterns of Enterprise Application Architecture

  • Software Architecture in Practice

Debugging and Refactoring

  • Debugging: The 9 Indispensable Rules …​ (2 weeks)

  • Refactoring: Improving the Design of Existing Code

  • Working Effectively with Legacy Code

System and network programming

Other important CS books and topics

  • Information Systems for Business and Beyond

  • The Passionate Programmer

  • The Clean Coder: A Code of Conduct for Professional Programmers

  • The Pragmatic Programmer: From Journeyman to Master (https://blog.codinghorror.com/a-pragmatic-quick-reference/)

  • Programming Pearls

  • Joel on Software: And on Diverse and Occasionally…​

  • Code Complete 2: A Practical Handbook of Software Construction

  • The Mythical Man-Month

  • Rapid Development: Taming Wild Software Schedules

  • Regular Expressions Cookbook: Detailed Solutions in Eight Programming Languages (+https://regexone.com/)


Computer Science

Scheme & Common lisp

  • The Little Schemer - Introduction to Scheme and functional programming in general

  • Practical Common Lisp - A good introduction to Lisp with several examples of practical use.

  • Common Lisp Recipes

Programming

  1. Structure and Interpretation of Computer Programs (hard)

  2. Concepts, Techniques, and Models of Computer Programming (harder)

  3. Elements of Programming (math, hard but very rewarding)

Computer Architecture

  1. Code: The Hidden Language of Computer Hardware and Software

  2. But how Do it Know? The Basic Principles of Computers for Everyone

  3. The Elements of Computing Systems

Operating systems

  1. Operating System Concepts 10th ed (the “Dinosaur book”)

  2. Operating Systems: Design and Implementation (Tanenbaum)

  3. Modern Operating Systems (Tanenbaum)

  4. Operating Systems: Three Easy Pieces

  5. The Design and Implementation of the FreeBSD Operating System

Databases

  1. Architecture of a Database System (Hellerstein, Stonebraker, Hamilton)

  2. Readings in Database Systems (databases Red Book)

  3. Database Management Systems (R.R.)

  4. Data and Reality (w kent)

Data Structures & Algorithms

Intro
  1. Grokking Algorithms

  2. Problem Solving with Algorithms and Data Structures using Python

Medium
  1. The Algorithm Design Manual (job interview prep)

  2. Open Data Structures (morin) (c++/java)

Advanced (rigorous books)
  1. Sedgewick: Algorithms (4th Edition) (java)

  2. Introduction to algorithms

  3. The Art of Computer Programming

Distributed Systems

  1. Distributed Systems (Tanenbaum, Steen)

  2. Designing Data-Intensive Applications (Kleppmann)

  3. Distributed Systems Reading Group

Mathematics

  1. discrete mathematics www.cs.elte.hu/~lovasz/dmbook.ps

  2. Mathematics for Computer Science link

  3. Introduction to Linear Algebra, Fifth Edition

  4. Esence of Linear algebra (3blue2brown)

Compilers/Interpreters

  1. Let’s Build a Compiler, by Jack Crenshaw

  2. A Nanopass Framework for Compiler Education

  3. Language Implementation Patterns (less theory-ey version of dragon book)

  4. LLVM Tutorial

  5. Programming Languages: Application and Interpretation

  6. Compilers: Principles, Techniques & Tools (the dragon book) (skim/cherry-pick!)


Security

So You Want To Break Shit
  • Learn C. Learn it well. That will give you the level of understanding you need of the way the computer works.

  • Learn Python or Ruby. If you can automate something, do it. Computers are better than you at repeatable tasks.

  • Learn x86 assembly (to start). Write some C code, compile it down to assembly, and read it; understand how the two map to each other. Get your friends to write stuff for you and decompile it back to C, and have them check your work.

  • Learn the basics (at least) of web development. Understand how web applications work, understand the constraints it puts in place, understand the interactions between the client and server.

  • Internalize the OWASP Top 10. Understand and be able to recognize and mitigate XSS, SQL injection, command injection, arbitrary file reads/writes, etc.

  • Grab old versions of open source software and rediscover known vulnerabilities. Grab the latest versions of open source software and discover new ones. Start off looking for simple things, and know it well.

  • Reverse-engineer network protocols. Pick a game, write a server emulator for it. This is a great way to use all your skills up to this point. It’s also a lot of fun (it’s how I cut my teeth).

  • Write a debugger. Understand the interaction between hardware, the kernel, and userland.

  • Understand, understand, understand. Ask questions. Ask a lot of questions. In my opinion, the key to security is always asking "Why?"

These are in no particular order, and none of these is more important than the other. It also jumps all over the place, perhaps because that’s what I do myself; I may be breaking a web app one day and reversing some hardware the next. But these are things I feel are important, and will give you some direction.

Resources
  • Foundations of Security: What Every Programmer Needs to Know ?

  • OWASP, GCIH, GCIA, SANs, OSCP are all well respected.

  • The Web Application Hacker’s Handbook 2nd Edition - Gives a very good overview and is a good place to start.

  • The Hacker Playbook 3: Practical Guide To Penetration Testing - #3 just came out. Haven’t gone through my copy yet, but I’ve heard good things.

  • RTFM - Red Team Field Manual - Nice to have, quick reference guide

  • BTFM - Blue Team Field Manual - Like the above, but for the good guys ;)

  • Hacking, 2nd edition

  • pick area of interest rom https://nostarch.com/catalog/security

  • follow smart security people on Twitter, which is the defacto medium for information security discussion

  • read publicly disclosed bug bounty reports on Hackerone and Bugcrowd

Other
  • 2FA is just something you know and something you have. I have a Yubikey with a RSA authorisation key (+enc and signing keys too, but that’s irrelevant) in it that I’ve hooked ssh-agent and GPG to. It is the only key accepted by my servers. Obviously, disable password login. The key has a password, which is the something I know.


Mobile App Dev

Android and Kotlin
Dart and flutter

?


Business

Books

Building a company
  • The Startup Owner’s Manual

  • Startup playbook (ycomb)

  • How to Start a Startup - Sam Altman

  • The Personal MBA - by Josh Kaufman (overview of everything you need to know)

  • Zero to One by Peter Thiel (monopolysim, phylosophy)

  • The Entrepreneur Roller Coaster - by Darren Hardy (must read)

  • E-Myth Revisited - by Michael Gerber (personal business view, org, planing, strategy)

  • The Hard Thing About Hard Things by Ben Horowitz (essential advice on building and running a startup)

  • The Art of the Start 2.0 (handbook on everything)

  • The Art of Profitability - by Adrian Slywotzky (different models of proffitability - profit-sources, single sitting)

  • Start Small, Stay Small - by Rob Walling and Mike Taber (on being micropreneur)

  • Rework by Jason Fried (stop talking and start working, single sitting)

  • anything you want by Derek S (manual for running a business, single sitting)

  • Running Lean: Iterate from Plan A to a Plan That Works

  • Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers

Investing and finance
  • The Smartest Investment Book You’ll Ever Read - by Daniel R. Solin

  • The Investor’s Manifesto - by William J. Bernstein

  • The Innovator’s Solution - by Clayton Christensen (on technology change)

  • The Intelligent Investor by Benjamin Graham (stocks?, hard book)

Biographies and Stories
  • The Launch Pad: Inside Y Combinator, Silicon Valley’s Most Exclusive School for Startups

  • The Leap: A Memoir of Love and Madness in the Internet Gold Rush (e commerce template)

  • Startup: A Silicon Valley Adventure

  • Delivering Happiness: A Path to Profits, Passion, and Purpose (narrative on Profits; Profits and Passion)

  • Hackers and Painters: Big Ideas from the Computer Age (essays on intelligence, entrepreneurship, programming, and questioning norms)


Management

Books

Project (software oriented)
  • Agile Software Development by Robert C. Martin (read carefully, its important)

  • Scrum and XP from the Trenches (For developers who want to do agile, how to actually implement scrum)

  • Implementing Lean Software Development: From Concept to Cash

  • Project Management Body of Knowledge

  • Making Things Happen: Mastering Project Management

  • The Goal (theory of constraints and operations management)

  • Running Lean: Iterate from Plan A to a Plan That Works

Team
  • Team Geek: A Software Developers Guide to working well with others (For engineers who want to work successfully with other engineers)

  • Managing the unmanageable: Rules, Tools…​ (For lead developers, line managers and others that need to steer software developers as a team)

  • Difficult Conversation: How To Discuss What Matters Most (solve conflicts in a straightforward manner)

  • The Five Dysfunctions of a Team: A Leadership Fable (fix dysfunctional teams)

  • Peopleware: Productive Projects and Teams (couple days, how to treat people. Written for managers)

  • Creativity, Inc. by Ed Catmull ( managerial how-to on fostering creativity, productivity and work/ life balance in the office.)

  • Crucial conversations

  • How to Win Friends and Influence People by Dale Carnegie

  • The No Asshole Rule: Building a Civilized Workplace and Surviving One That Isn’t

  • The One Thing You Need to Know by Marcus Buckingham

  • Act Like a Leader, Think Like a Leader by Herminia Ibarra

  • Drive - by Daniel Pink (what drives people to do the work)

  • Influence - by Robert Cialdini (psychology of persuasion)

  • Predictably Irrational - by Dan Ariely

  • The Ultimate Sales Machine - by Chet Holmes (like e myth, how to turn it into real results)

  • Tribes - by Seth Godin (what it takes to organize and mobilize groups of people)


Notebooks

Notes

TODO: sort all of this better.

General advice

Studying and working basics
  • Eliminate distractions, don’t keep checking email, switching songs on youtube etc.

  • Don’t overwork yourself - recognize when you need to take breaks.

  • Simplify things as much as you can

  • When you’re working through a book, type out the example code and do all the exercises.

  • if you build something document it! take notes about what you have learned.

  • Know what your code does and why it does it. Do not commit guesswork into your repository.

  • On a bigger projects include a build process, automated tests, a README.md file that clearly explains the project, and practice using meaningful commit messages. Use each project to demonstrate that you can deliver an optimized product. Implement tests, use your build process to minimize assets, use a linter, use style guides and naming conventions etc

Time
  • Slicing. Take a big project, cut it into thin slices, and rearrange the slices to suit your context. I can always slice projects finer and I can always find new permutations of the slices that meet different needs.

  • One thing at a time. We’re so focused on efficiency that we reduce the number of feedback cycles in an attempt to reduce overhead. This leads to difficult debugging situations whose expected cost is greater than the cycle overhead we avoided.

  • Make it run, make it right, make it fast. (Example of One Thing at a Time, Slicing, and Easy Changes)

  • Easy changes. When faced with a hard change, first make it easy (warning, this may be hard), then make the easy change. (e.g. slicing, one thing at a time, concentration, isolation). Example of slicing.

  • Concentration. If you need to change several elements, first rearrange the code so the change only needs to happen in one element.

  • Isolation. If you only need to change a part of an element, extract that part so the whole subelement changes.

  • Baseline Measurement. Start projects by measuring the current state of the world. This goes against our engineering instincts to start fixing things, but when you measure the baseline you will actually know whether you are fixing things.

Learning
  • Call your shot. Before you run code, predict out loud exactly what will happen.

  • Concrete hypotheses. When the program is misbehaving, articulate exactly what you think is wrong before making a change. If you have two or more hypotheses, find a differential diagnosis.

  • Remove extraneous detail. When reporting a bug, find the shortest repro steps. When isolating a bug, find the shortest test case. When using a new API, start from the most basic example. “All that stuff can’t possibly matter,” is an expensive assumption when it’s wrong. E.g. see a bug on mobile, reproduce it with curl

  • Multiple scales. Move between scales freely. Maybe this is a design problem, not a testing problem. Maybe it is a people problem, not a technology problem [cheating, this is always true].

Transcend Logic
  • Symmetry. Things that are almost the same can be divided into parts that are identical and parts that are clearly different.

  • Aesthetics. Beauty is a powerful gradient to climb. It is also a liberating gradient to flout (e.g. inlining a bunch of functions into one giant mess).

  • Rhythm. Waiting until the right moment preserves energy and avoids clutter. Act with intensity when the time comes to act.

  • Tradeoffs. All decisions are subject to tradeoffs. It’s more important to know what the decision depends on than it is to know which answer to pick today (or which answer you picked yesterday).

Risk
  • Fun list. When tangential ideas come, note them and get back to work quickly. Revisit this list when you’ve reached a stopping spot.

  • Feed Ideas. Ideas are like frightened little birds. If you scare them away they will stop coming around. When you have an idea, feed it a little. Invalidate it as quickly as you can, but from data not from a lack of self-esteem.

  • 80/15/5. Spend 80% of your time on low-risk/reasonable-payoff work. Spend 15% of your time on related high-risk/high-payoff work. Spend 5% of your time on things that tickle you, regardless of payoff. Teach the next generation to do your 80% job. By the time someone is ready to take over, one of your 15% experiments (or, less frequently, one of your 5% experiments) will have paid off and will become your new 80%. Repeat.

On achieving peak performance:
  • Turn off the computer. Grab a pencil and some paper. Sketch out your design. Review it with your peers. Then write the code.

  • Avoid gold plating - do only what is asked of you (in terms of requirements)

  • Understand business requirements and do it right the first time

  • Thoroughly understand your environment and tools

  • Become a fantastic typist, use keyboard shortcuts instead of the mouse

  • Take an iterative approach and build in sanity checks to ensure you are on the right path

  • Don’t reinvent the wheel, consider reusing past work and the work of others

  • Eliminate distractions, don’t keep checking email, looking outside, talking to coworkers, etc.

  • Don’t overwork yourself - recognize when you need to take breaks

  • Do a postmortem on the last project estimates vs. actual time taken and figure out where the extra time went.

  • simplify what you are doing so that it is absolutely as simple as possible

Traits of an experts
  • Identify patterns faster and successfully predict future events

  • Recognize anomalies - especially negative anomalies i.e. something didn’t happen that should - quickly take appropriate actions.

  • Identify leverage points within their architecture to solve new problems and deliver new features.

  • Make fine discriminations of events and data

  • Understand the tradeoffs and consequences of an option.

  • Recognize expertise in others and defer as many decisions as possible to that expertise.

  • Their ability to "context switch" when describing a situation to other experts vs novices vs non-particpants.

  • Skate where the puck is going, not where it is.

  • Rapidly Climb Learning Curves The ability to quickly learn enough about new subjects to be useful. New technologies or APIs; new algorithms; mathematical or statistical subjects; and most importantly, your problem domain. Some of this ability is a skill, "knowing how to learn", which covers google-fu, reading comprehension, prioritization, time management, etc. Some of this ability comes from having enough aggregate experience that new information just clicks into place like jigsaw puzzle pieces. Some of this ability comes from the confidence of having climbed enough seemingly insurmountable learning curves that this next one is "just another day at the office".

  • Understand The Customer The best engineers are half product managers. You can easily get a 10X improvement in productivity by building the right features and not building the wrong ones. Yes, professional product managers are great, but technical limitations often impact product design. A great engineer knows when to push back or suggest alternatives. This requires empathy not only for the customer, but for the product management team. This ability is also tightly coupled with #1 - you need to quickly come up to speed on the problem domain, whatever that may be. If you wan’t to be a great engineer, don’t study algorithms (until you need an algorithm, of course), study the particular business that you’re in.

Job

Career Advice (engineers)

90% of programming jobs are in creating Line of Business software: Economics 101: the price for anything (including you) is a function of the supply of it and demand for it. Let’s talk about the demand side first. Most software is not sold in boxes, available on the Internet, or downloaded from the App Store. Most software is boring one-off applications in corporations, under-girding every imaginable facet of the global economy. It tracks expenses, it optimizes shipping costs, it assists the accounting department in preparing projections, it helps design new widgets, it prices insurance policies, it flags orders for manual review by the fraud department, etc etc. Software solves business problems. Software often solves business problems despite being soul-crushingly boring and of minimal technical complexity. For example, consider an internal travel expense reporting form. Across a company with 2,000 employees, that might save 5,000 man-hours a year (at an average fully-loaded cost of $50 an hour) versus handling expenses on paper, for a savings of $250,000 a year. It does not matter to the company that the reporting form is the world’s simplest CRUD app, it only matters that it either saves the company costs or generates additional revenue.

There are companies which create software which actually gets used by customers, which describes almost everything that you probably think of when you think of software. It is unlikely that you will work at one unless you work towards making this happen. Even if you actually work at one, many of the programmers there do not work on customer-facing software, either.

Engineers are hired to create business value, not to program things: Businesses do things for irrational and political reasons all the time (see below), but in the main they converge on doing things which increase revenue or reduce costs. Status in well-run businesses generally is awarded to people who successfully take credit for doing one of these things. (That can, but does not necessarily, entail actually doing them.) The person who has decided to bring on one more engineer is not doing it because they love having a geek around the room, they are doing it because adding the geek allows them to complete a project (or projects) which will add revenue or decrease costs. Producing beautiful software is not a goal. Solving complex technical problems is not a goal. Writing bug-free code is not a goal. Using sexy programming languages is not a goal. Add revenue. Reduce costs. Those are your only goals.

Peter Drucker — you haven’t heard of him, but he is a prophet among people who sign checks — came up with the terms Profit Center and Cost Center. Profit Centers are the part of an organization that bring in the bacon: partners at law firms, sales at enterprise software companies, “masters of the universe” on Wall Street, etc etc. Cost Centers are, well, everybody else. You really want to be attached to Profit Centers because it will bring you higher wages, more respect, and greater opportunities for everything of value to you. It isn’t hard: a bright high schooler, given a paragraph-long description of a business, can usually identify where the Profit Center is. If you want to work there, work for that. If you can’t, either a) work elsewhere or b) engineer your transfer after joining the company.

Engineers in particular are usually very highly paid Cost Centers, which sets MBA’s optimization antennae to twitching. This is what brings us wonderful ideas like outsourcing, which is “Let’s replace really expensive Cost Centers who do some magic which we kinda need but don’t really care about with less expensive Cost Centers in a lower wage country”. (Quick sidenote: You can absolutely ignore outsourcing as a career threat if you read the rest of this guide.) Nobody ever outsources Profit Centers. Attempting to do so would be the setup for MBA humor. It’s like suggesting replacing your source control system with a bunch of copies maintained on floppy disks.

Don’t call yourself a programmer: “Programmer” sounds like “anomalously high-cost peon who types some mumbo-jumbo into some other mumbo-jumbo.” If you call yourself a programmer, someone is already working on a way to get you fired. You know Salesforce, widely perceived among engineers to be a Software as a Services company? Their motto and sales point is “No Software”, which conveys to their actual customers “You know those programmers you have working on your internal systems? If you used Salesforce, you could fire half of them and pocket part of the difference in your bonus.” (There’s nothing wrong with this, by the way. You’re in the business of unemploying people. If you think that is unfair, go back to school and study something that doesn’t matter.)

Instead, describe yourself by what you have accomplished for previously employers vis-a-vis increasing revenues or reducing costs. If you have not had the opportunity to do this yet, describe things which suggest you have the ability to increase revenue or reduce costs, or ideas to do so.

There are many varieties of well-paid professionals who sling code but do not describe themselves as slinging code for a living. Quants on Wall Street are the first and best-known example: they use computers and math as a lever to make high-consequence decisions better and faster than an unaided human could, and the punchline to those decisions is “our firm make billions of dollars.” Successful quants make more in bonuses in a good year than many equivalently talented engineers will earn in a decade or lifetime.

Similarly, even though you might think Google sounds like a programmer-friendly company, there are programmers and then there’s the people who are closely tied to 1% improvements in AdWords click-through rates. (Hint: provably worth billions of dollars.) I recently stumbled across a web-page from the guy whose professional bio is “wrote the backend billing code that 97% of Google’s revenue passes through.” He’s now an angel investor (a polite synonym for “rich”).

You are not defined by your chosen software stack: I recently asked via Twitter what young engineers wanted to know about careers. Many asked how to know what programming language or stack to study. It doesn’t matter. There you go.

Do Java programmers make more money than .NET programmers? Anyone describing themselves as either a Java programmer or .NET programmer has already lost, because a) they’re a programmer (you’re not, see above) and b) they’re making themselves non-hireable for most programming jobs. In the real world, picking up a new language takes a few weeks of effort and after 6 to 12 months nobody will ever notice you haven’t been doing that one for your entire career. I did back-end Big Freaking Java Web Application development as recently as March 2010. Trust me, nobody cares about that. If a Python shop was looking for somebody technical to make them a pile of money, the fact that I’ve never written a line of Python would not get held against me.

Talented engineers are rare — vastly rarer than opportunities to use them — and it is a seller’s market for talent right now in almost every facet of the field. Everybody at Matasano uses Ruby. If you don’t, but are a good engineer, they’ll hire you anyway. (A good engineer has a track record of — repeat after me — increasing revenue or decreasing costs.) Much of Fog Creek uses the Microsoft Stack. I can’t even spell ASP.NET and they’d still hire me.

There are companies with broken HR policies where lack of a buzzword means you won’t be selected. You don’t want to work for them, but if you really do, you can add the relevant buzzword to your resume for the costs of a few nights and weekends, or by controlling technology choices at your current job in such a manner that in advances your career interests. Want to get trained on Ruby at a .NET shop? Implement a one-off project in Ruby. Bam, you are now a professional Ruby programmer — you coded Ruby and you took money for it. (You laugh? I did this at a Java shop. The one-off Ruby project made the company $30,000. My boss was, predictably, quite happy and never even asked what produced the deliverable.)

Co-workers and bosses are not usually your friends: You will spend a lot of time with co-workers. You may eventually become close friends with some of them, but in general, you will move on in three years and aside from maintaining cordial relations you will not go out of your way to invite them over to dinner. They will treat you in exactly the same way. You should be a good person to everyone you meet — it is the moral thing to do, and as a sidenote will really help your networking — but do not be under the delusion that everyone is your friend.

For example, at a job interview, even if you are talking to an affable 28 year old who feels like a slightly older version of you he is in a transaction. You are not his friend, you are an input for an industrial process which he is trying to buy for the company at the lowest price. That banter about World of Warcraft is just establishing a professional rapport, but he will (perfectly ethically) attempt to do things that none of your actual friends would ever do, like try to talk you down several thousand dollars in salary or guilt-trip you into spending more time with the company when you could be spending time with your actual friends. You will have other coworkers who — affably and ethically — will suggest things which go against your interests, from “I should get credit for that project you just did” (probably not phrased in so many words) to “We should do this thing which advances my professional growth goals rather than yours.” Don’t be surprised when this happens.

You radically overestimate the average skill of the competition because of the crowd you hang around with: Many people already successfully employed as senior engineers cannot actually implement FizzBuzz. Just read it and weep. Key takeaway: you probably are good enough to work at that company you think you’re not good enough for. They hire better mortals, but they still hire mortals.

“Read ad. Send in resume. Go to job interview. Receive offer.” is the exception, not the typical case, for getting employment: Most jobs are never available publicly, just like most worthwhile candidates are not available publicly (see here). Information about the position travels at approximately the speed of beer, sometimes lubricated by email. The decisionmaker at a company knows he needs someone. He tells his friends and business contacts. One of them knows someone — family, a roommate from college, someone they met at a conference, an ex-colleague, whatever. Introductions are made, a meeting happens, and they achieve agreement in principle on the job offer. Then the resume/HR department/formal offer dance comes about.

This is disproportionately true of jobs you actually want to get. “First employee at a successful startup” has a certain cachet for a lot of geeks, and virtually none of those got placed by sending in a cover letter to an HR department, in part because two-man startups don’t have enough scar tissue to form HR departments yet. (P.S. You probably don’t want to be first employee for a startup. Be the last co-founder instead.) Want to get a job at Google? They have a formal process for giving you a leg up because a Googler likes you. (They also have multiple informal ways for a Googler who likes you an awful lot to short-circuit that process. One example: buy the company you work for. When you have a couple of billion lying around you have many interesting options for solving problems.)

There are many reasons why most hiring happens privately. One is that publicly visible job offers get spammed by hundreds of resumes (particularly in this economy) from people who are stunningly inappropriate for the position. The other is that other companies are so bad at hiring that, if you don’t have close personal knowledge about the candidate, you might accidentally hire a non-FizzBuzzer.

Networking: it isn’t just for TCP packets: Networking just means a) meeting people who at some point can do things for you (or vice versa) and b) making a favorable impression on them.

There are many places to meet people. Events in your industry, such as conferences or academic symposia which get seen by non-academics, are one. User groups are another. Keep in mind that user groups draw a very different crowd than industry conferences and optimize accordingly.

Strive to help people. It is the right thing to do, and people are keenly aware of who have in the past given them or theirs favors. If you ever can’t help someone but know someone who can, pass them to the appropriate person with a recommendation. If you do this right, two people will be happy with you and favorably disposed to helping you out in the future.

You can meet people over the Internet (oh God, can you), but something in our monkey brains makes in-the-flesh meeting a bigger thing. I’ve Internet-met a great many people who I’ve then gone on to meet in real life. The physical handshake is a major step up in the relationship, even when Internet-meeting lead to very consequential things like “Made them a lot of money through good advice.” Definitely blog and participate on your industry-appropriate watering holes like HN, but make it out to the meetups for it.

Academia is not like the real world: Your GPA largely doesn’t matter (modulo one high profile exception: a multinational advertising firm). To the extent that it does matter, it only determines whether your resume gets selected for job interviews. If you’re reading the rest of this, you know that your resume isn’t the primary way to get job interviews, so don’t spend huge amount of efforts optimizing something that you either have sufficiently optimized already (since you’ll get the same amount of interviews at 3.96 as you will at 3.8) or that you don’t need at all (since you’ll get job interviews because you’re competent at asking the right people to have coffee with you).

Your major and minor don’t matter. Most decisionmakers in industry couldn’t tell the difference between a major in Computer Science and a major in Mathematics if they tried. I was once reduced to tears because a minor academic snafu threatened my ability to get a Bachelor of Science with a major in Computer Science, which my advisor told me was more prestigious than a Bachelor of Science in Computer Science. Academia cares about distinctions like that. The real world does not.

Your professors might understand how the academic job market works (short story: it is ridiculously inefficient in engineering and fubared beyond mortal comprehension in English) but they often have quixotic understandings of how the real world works. For example, they may push you to get extra degrees because a) it sounds like a good idea to them and b) they enjoy having research-producing peons who work for ramen. Remember, market wages for people capable of producing research are $80~100k+ in your field. That buys an awful lot of ramen.

The prof in charge of my research project offered me a spot in his lab, a tuition waiver, and a whole $12,000 dollars as a stipend if I would commit 4~6 years to him. That’s a great deal if, and only if, you have recently immigrated from a low-wage country and need someone to intervene with the government to get you a visa.

If you really like the atmosphere at universities, that is cool. Put a backpack on and you can walk into any building at any university in the United States any time you want. Backpacks are a lot cheaper than working in academia. You can lead the life of the mind in industry, too — and enjoy less politics and better pay. You can even get published in journals, if that floats your boat. (After you’ve escaped the mind-warping miasma of academia, you might rightfully question whether Published In A Journal is really personally or societally significant as opposed to close approximations like Wrote A Blog Post And Showed It To Smart People.)

How much money do engineers make?

Wrong question. The right question is “What kind of offers do engineers routinely work for?”, because salary is one of many levers that people can use to motivate you. The answer to this is, less than helpfully, “Offers are all over the map.”

In general, big companies pay more (money, benefits, etc) than startups. Engineers with high perceived value make more than those with low perceived value. Senior engineers make more than junior engineers. People working in high-cost areas make more than people in low-cost areas. People who are skilled in negotiation make more than those who are not.

We have strong cultural training to not ask about salary, ever. This is not universal. In many cultures, professional contexts are a perfectly appropriate time to discuss money. (If you were a middle class Japanese man, you could reasonably be expected to reveal your exact salary to a 2nd date, anyone from your soccer club, or the guy who makes your sushi. If you owned a company, you’d probably be cagey about your net worth but you’d talk about employee salaries the way programmers talk about compilers — quite frequently, without being embarrassed.) If I were a Marxist academic or a conspiracy theorist, I might think that this bit of middle class American culture was specifically engineered to be in the interests of employers and against the interests of employees. Prior to a discussion of salary at any particular target employer, you should speak to someone who works there in a similar situation and ask about the salary range for the position. It is <%= Date.today.year %>; you can find these people online. (LinkedIn, Facebook, Twitter, and your (non-graph-database) social networks are all good to lean on.)

Anyhow. Engineers are routinely offered a suite of benefits. It is worth worrying, in the United States, about health insurance (traditionally, you get it and your employer foots most or all of the costs) and your retirement program, which is some variant of “we will match contributions to your 401k up to X% of salary.” The value of that is easy to calculate: X% of salary. (It is free money, so always max out your IRA up to the employer match. Put it in index funds and forget about it for 40 years.)

There are other benefits like “free soda”, “catered lunches”, “free programming books”, etc. These are social signals more than anything else. When I say that I’m going to buy you soda, that says a specific thing about how I run my workplace, who I expect to work for me, and how I expect to treat them. (It says “I like to move the behavior of unsophisticated young engineers by making this job seem fun by buying 20 cent cans of soda, saving myself tens of thousands in compensation while simultaneously encouraging them to ruin their health.” And I like soda.) Read social signals and react appropriately — someone who signals that, e.g., employee education is worth paying money for might very well be a great company to work for — but don’t give up huge amounts of compensation in return for perks that you could trivially buy.

How do I become better at negotiation?

a) Remember you’re selling the solution to a business need (raise revenue or decrease costs) rather than programming skill or your beautiful face.

b) Negotiate aggressively with appropriate confidence, like the ethical professional you are. It is what your counterparty is probably doing. You’re aiming for a mutual beneficial offer, not for saying Yes every time they say something.

c) “What is your previous salary?” is employer-speak for “Please give me reasons to pay you less money.” Answer appropriately.

d) Always have a counteroffer. Be comfortable counteroffering around axes you care about other than money. If they can’t go higher on salary then talk about vacation instead.

e) The only time to ever discuss salary is after you have reached agreement in principle that they will hire you if you can strike a mutually beneficial deal. This is late in the process after they have invested a lot of time and money in you, specifically, not at the interview. Remember that there are large costs associated with them saying “No, we can’t make that work” and, appropriately, they will probably not scuttle the deal over comparatively small issues which matter quite a bit to you, like e.g. taking their offer and countering for that plus a few thousand bucks then sticking to it.

f) Read a book. Many have been written about negotiation. I like Getting To Yes. It is a little disconcerting that negotiation skills are worth thousands of dollars per year for your entire career but engineers think that directed effort to study them is crazy when that could be applied to trivialities about a technology that briefly caught their fancy.

How to value an equity grant:

Roll d100. (Not the right kind of geek? Sorry. rand(100) then.)

0~70: Your equity grant is worth nothing.

71~94: Your equity grant is worth a lump sum of money which makes you about as much money as you gave up working for the startup, instead of working for a megacorp at a higher salary with better benefits.

95~99: Your equity grant is a lifechanging amount of money. You won’t feel rich — you’re not the richest person you know, because many of the people you spent the last several years with are now richer than you by definition — but your family will never again give you grief for not having gone into $FAVORED_FIELD like a proper $YOUR_INGROUP.

100: You worked at the next Google, and are rich beyond the dreams of avarice. Congratulations.

Perceptive readers will note that 100 does not actually show up on a d100 or rand(100).

Why are you so negative about equity grants?

Because you radically overestimate the likelihood that your startup will succeed and radically overestimate the portion of the pie that will be allocated to you if the startup succeeds. Read about dilution and liquidation preferences on Hacker News or Venture Hacks, then remember that there are people who know more about negotiating deals than you know about programming and imagine what you could do to a program if there were several hundred million on the line.

Are startups great for your career as a fresh graduate?

The high-percentage outcome is you work really hard for the next couple of years, fail ingloriously, and then be jobless and looking to get into another startup. If you really wanted to get into a startup two years out of school, you could also just go work at a megacorp for the next two years, earn a bit of money, then take your warchest, domain knowledge, and contacts and found one.

Working at a startup, you tend to meet people doing startups. Most of them will not be able to hire you in two years. Working at a large corporation, you tend to meet other people in large corporations in your area. Many of them either will be able to hire you or will have the ear of someone able to hire you in two years.

So would you recommend working at a startup? Working in a startup is a career path but, more than that, it is a lifestyle choice. This is similar to working in investment banking or academia. Those are three very different lifestyles. Many people will attempt to sell you those lifestyles as being in your interests, for their own reasons. If you genuinely would enjoy that lifestyle, go nuts. If you only enjoy certain bits of it, remember that many things are available a la carte if you really want them. For example, if you want to work on cutting-edge technology but also want to see your kids at 5:30 PM, you can work on cutting-edge technology at many, many, many megacorps.

(Yeah, really. If it creates value for them, heck yes, they’ll invest in it. They’ll also invest in a lot of CRUD apps, but then again, so do startups — they just market making CRUD apps better than most megacorps do. The first hour of the Social Network is about making a CRUD app seem like sexy, the second is a Lifetime drama about a divorce improbably involving two heterosexual men.)

Your most important professional skill is communication: Remember engineers are not hired to create programs and how they are hired to create business value? The dominant quality which gets you jobs is the ability to give people the perception that you will create value. This is not necessarily coextensive with ability to create value.

Some of the best programmers I know are pathologically incapable of carrying on a conversation. People disproportionately a) wouldn’t want to work with them or b) will underestimate their value-creation ability because they gain insight into that ability through conversation and the person just doesn’t implement that protocol. Conversely, people routinely assume that I am among the best programmers they know entirely because a) there exists observable evidence that I can program and b) I write and speak really, really well.

(Once upon a time I would have described myself as “Slightly below average” in programming skill. I have since learned that I had a radically skewed impression of the skill distribution, that programming skill is not what people actually optimize for, and that modesty is against my interests. These days if you ask me how good of a programmer I am I will start telling you stories about how I have programmed systems which helped millions of kids learn to read or which provably made companies millions. The question of where I am on the bell curve matters to no one, so why bother worrying about it?)

Communication is a skill. Practice it: you will get better. One key sub-skill is being able to quickly, concisely, and confidently explain how you create value to someone who is not an expert in your field and who does not have a priori reasons to love you. If when you attempt to do this technical buzzwords keep coming up (“Reduced 99th percentile query times by 200 ms by optimizing indexes on…”), take them out and try again. You should be able to explain what you do to a bright 8 year old, the CFO of your company, or a programmer in a different specialty, at whatever the appropriate level of abstraction is.

You will often be called to do Enterprise Sales and other stuff you got into engineering to avoid: Enterprise Sales is going into a corporation and trying to convince them to spend six or seven figures on buying a system which will either improve their revenue or reduce costs. Every job interview you will ever have is Enterprise Sales. Politics, relationships, and communication skills matter a heck of a lot, technical reality not quite so much.

When you have meetings with coworkers and are attempting to convince them to implement your suggestions, you will also be doing Enterprise Sales. If getting stuff done is your job description, then convincing people to get stuff done is a core job skill for you. Spend appropriate effort on getting good at it. This means being able to communicate effectively in memos, emails, conversations, meetings, and PowerPoint (when appropriate). It means understanding how to make a business case for a technological initiative. It means knowing that sometimes you will make technological sacrifices in pursuit of business objectives and that this is the right call.

Modesty is not a career-enhancing character trait: Many engineers have self-confidence issues (hello, self). Many also come from upbringings where modesty with regards to one’s accomplishments is culturally celebrated. American businesses largely do not value modesty about one’s accomplishments. The right tone to aim for in interviews, interactions with other people, and life is closer to “restrained, confident professionalism.”

If you are part of a team effort and the team effort succeeds, the right note to hit is not “I owe it all to my team” unless your position is such that everyone will understand you are lying to be modest. Try for “It was a privilege to assist my team by leading their efforts with regards to $YOUR_SPECIALTY.” Say it in a mirror a thousand times until you can say it with a straight face. You might feel like you’re overstating your accomplishments. Screw that. Someone who claims to Lead Efforts To Optimize Production while having the title Sandwich Artist is overstating their accomplishments. You are an engineer. You work magic which makes people’s lives better. If you were in charge of the database specifically on an important project involving people then heck yes you lead the database effort which was crucial for the success of the project. This is how the game is played. If you feel poorly about it, you’re like a batter who feels poorly about stealing bases in baseball: you’re not morally superior, you’re just playing poorly

All business decisions are ultimately made by one or a handful of multi-cellular organisms closely related to chimpanzees, not by rules or by algorithms: People are people. Social grooming is a really important skill. People will often back suggestions by friends because they are friends, even when other suggestions might actually be better. People will often be favoritably disposed to people they have broken bread with. (There is a business book called Never Eat Alone. It might be worth reading, but that title is whatever the antonym of deceptive advertising is.) People routinely favor people who they think are like them over people they think are not like them. (This can be good, neutral, or invidious. Accepting that it happens is the first step to profitably exploiting it.)

Actual grooming is at least moderately important, too, because people are hilariously easy to hack by expedients such as dressing appropriately for the situation, maintaining a professional appearance, speaking in a confident tone of voice, etc. Your business suit will probably cost about as much as a computer monitor. You only need it once in a blue moon, but when you need it you’ll be really, really, really glad that you have it. Take my word for it, if I wear everyday casual when I visit e.g. City Hall I get treated like a hapless awkward twenty-something, if I wear the suit I get treated like the CEO of a multinational company. I’m actually the awkward twenty-something CEO of a multinational company, but I get to pick which side to emphasize when I want favorable treatment from a bureaucrat.

At the end of the day, your life happiness will not be dominated by your career. Either talk to older people or trust the social scientists who have: family, faith, hobbies, etc etc generally swamp career achievements and money in terms of things which actually produce happiness. Optimize appropriately. Your career is important, and right now it might seem like the most important thing in your life, but odds are that is not what you’ll believe forever. Work to live, don’t live to work.

Finding a job

Finding c++ job: In the end the company I work for seemed really cool, but they had no "careers" or "work here" section on their web site. So I cold called their sales department and asked if I could talk to an engineer just to ask some questions as a student. Got an email back a few days later from the head of engineering and the rest is history. Most of the companies in the industry are small and don’t actively recruit as far as I can tell. Makes the networking/cold call part that much more necessary.

Salary negotiation

Rich, successful people negotiate.

Get into the habit of seeing employees like employers see them: in terms of fully-loaded costs. To hire someone you need to pay for their salary, true, but you also have taxes, a benefits package, employer contributions to retirement, healthcare, that free soda your HR department loves mentioning in the job ads, and what have you. The fully-loaded costs of employees are much higher than their salary: exactly how much higher depends on your locality’s laws, your benefits package, and a bunch of other HR administrivia, but a reasonable guesstimate is between 150% and 200% of their salary.

Companies are not sensitive to small differences in employee wages because employees are so darned expensive anyhow. You see $5,000 and think “Holy cow, even after taxes that’s a whole new vacation. Five thousand dollars. Five thousand dollars. It would be so very, very greedy of me to ask for five thousand whole dollars.” The HR department sees $5,000 and thinks “Meh, even after we kick in the extra taxes, that is only about 3% of their fully-loaded cost for this year anyhow, or seven hundredths of one percent of that team’s hiring budget. I wonder if the cafeteria has carrot cake today?”

Keep in mind that in startup world making a new hire is a big commitment, but they still have a lot of flexibility on the details because the details do not shave months off of their runway, think equity.

Internalize this: everyone in this discussion is a businessman. Bob, the HR person is just spending a part of hiring budget, he doesn’t really care about you or the company. If the deal makes economic sense, it will happen. If it doesn’t, firm handshakes will be exchanged, non-specific promises will be uttered, and everyone will forget about this discussion in a matter of hours. You will not be blackballed for negotiating. Bob couldn’t care less and, even if he did care, he has better things to do with his time than worry about a candidate he didn’t hire. Bob is working through a list of a dozen people right now, and his manager Dave is being such a hard case about that project’s schedule, and he’s not sure he can make his daughter’s piano recital, and the cafeteria’s carrot cake was a little dry. Bob is far, far less invested in this negotiation than you are.

Many people think job searches go something like this:

  1. See ad for job on Monster.com

  2. Send in a resume.

  3. Get an interview.

  4. Get asked for salary requirements.

  5. Get offered your salary requirement plus 5%.

  6. Try to negotiate that offer, if you can bring yourself to.

This is an effective strategy for job searching if you enjoy alternating bouts of being unemployed, being poorly compensated, and then treated like a disposable peon. (I served three years as a disposable peon in a Japanese megacorp and might be projecting a tad bit here. Regardless, my loss is your gain.)

You will have much, much better results if your job search looks something more like:

  1. (Optional but recommended) Establish a reputation in your field as someone who delivers measurable results vis-a-vis improving revenue or reducing costs.

  2. Have a hiring manager talk with you, specifically, about an opening that they want you, specifically, to fill.

  3. alk informally (and then possibly formally) and come to the conclusion that this would be a great thing if both sides could come to a mutually fulfilling offer.

  4. Let them take a stab at what that mutually fulfilling offer would look like.

  5. Suggest ways that they could improve it such that the path is cleared for you doing that voodoo that you do so well to improve their revenue and/or reduce their costs.

  6. (Optional) Give the guy hiring you a resume to send to HR, for their records. Nobody will read it, because resumes are an institution created to mean that no one has to read resumes. Since no one will read it, we put it in the process where it literally doesn’t matter whether it happens or not, because if you had your job offer contingent on a document that everyone knows no one reads, that would be pretty effing stupid now wouldn’t it.

Only negotiate salary after you have agreement in principle from someone with hiring authority that, if a mutually acceptable compensation can be agreed upon, you will be hired.

The second implication is that the inner serf worrying “If I even attempt to negotiate this, the deal will fall through” is worrying for nothing. They’ve got thousands invested in this discussion by this point. They want you. The absolute worst outcome of negotiating an offer in good faith is that you will get exactly the contents of that offer. Let me say that again for emphasis: negotiating never makes (worthwhile) offers worse.

In this wide world I’m sure you can find a company who still makes exploding offers, where you get one yay-or-nay and then the offer is gone. You have a simple recourse to them: refuse them and deal with people who are willing to be professionals. You’re not a peasant. Don’t act like one.

This also means you do not start negotiating until you already have a Yes-If. (Yes-If we agree on terms.) Do not start negotiating from No-But. (No-But we might hire you anyway if you’re really, really effing cheap.)

The ideal resolution to the job interview is for both sides to be optimistic about the arrangement, and then you close with a warm handshake and “I look forward to receiving your offer by, oh, would tomorrow be enough time for you to run the numbers?”

You then have a high likelihood of doing your salary negotiation over email, which is likely to your advantage versus doing it in real time. Email gives you arbitrary time to prepare your responses. Especially for engineers, you are likely less disadvantaged by email than you are by having an experienced negotiator talking to you.

The First Rule Is What Everyone Tells You It Is: Never Give A Number First

Case 1

Bob: “I really need a number to move the process forward.”

What you should think: “You’re lying to me to attempt to get me to compromise my negotiating position.”

What you should say: “I’m more concerned at the moment with talking to you about discovering whether we’re a mutual fit. If we’re a great fit, then I can be flexible on the numbers with you and you can be flexible on the numbers with me. If we’re not a great fit, then the numbers are ultimately irrelevant, because your company only hires A players and I only work at roles I would be an A player at.”

Case 2

Bob: “This form needs a number.”

What you should think: “You’re lying to me to attempt to get me to compromise my negotiating position.”

What you should say: “Give me git access and I’ll fix it in a jiffy! both people laugh No, seriously, speaking, I’m more concerned at the moment with discovering whether we’re a mutual fit… Oh, it’s physically impossible? Put in $1 then to get the ball rolling, and we’ll circle back to this later.”

Case 3

Bob: “We want to figure out whether you’re an appropriate candidate for the position.”

What you should think: “You’re lying to me to attempt to get me to compromise my negotiating position.”

What you should say: “It’s so important to me that this is a good mutual fit for us. Let’s talk about why I’m a great fit for this position: I know you’re concerned about $FILL_IN_THE_BLANK. In addition to my previous successes doing it, I have some great ideas for what I’d do about that if I was working at your company. Would you like to drill into those or is there another job area you’re more concerned about to start with?”

Case 4

Bob: “I’m sorry, great try at a dodge there, but I just can’t go forward without a number.”

What you should think: “You’re lying to me to attempt to get me to compromise my negotiating position.”

What you should say (if you’re an engineer): “Well, you know, I would hate to have to walk away from the negotiation over this. Working with your company looked it would have been such a wonderful opportunity. I hear the hiring market is super-tight right now, would you like me to introduce you to other candidates? Maybe we can shave a couple of months off of you filling this position.”

What you should say (if you’re not an engineer): “Damn, I guess I should have studied engineering.”

What you should say (if you’re a little put out by that comment): “Well, you know, salary is only one component of the total compensation package. In terms of total compensation, we’re probably looking at something like $FILL_IN_NUMBER_HERE.” (Suggested calculation: take the package value from your last company and add 5~10%. If you don’t know how to calculate the value of your compensation package, learn that, but as a rough guesstimate salary + 30 ~ 50% for full-time employees in professional roles and the multiplier tends to scale up as your base salary scales up.)

Listen To What People Tell You. Repeat It Back To Them.

You know what people find persuasive? Their own words. People love their own words. When you talk to them, you should use their own words. Seriously, watch the eyes light up.

Did the solicitation for the job say “We are seeking someone with strong skills at scaling traffic in a fast-moving environment”? Pick out the key words. Scaling traffic. Fast-moving environment. “Scaling traffic” doesn’t sound like how I’d phrase it if I were writing or speaking for myself, but if you’ve just described your need to me as scaling traffic, by golly I will tell you how great I am at scaling traffic. Reinterpret or rephrase the (true!) bits of your own story such that it fits the narrative framework which they have conveniently told you that they are going to respond to. Did you previously work at a small business which was unencumbered by lots of process? Sounds like a fast-moving environment, right? Call it exactly that, then.

Take notes during job interviews and salary negotiations. It’s easy: go to the convenience store before the job interview, buy a writing instrument and a $1 notebook, jot down occasional notes when appropriate.

Do you know anyone who you’ve ever thought “Man, I thought they were competent, but then it turned out they had a notebook so I had to write them off?” No. Taking notes says “I’m attentive and detail-oriented and I care about what you say.” (Make sure you can take notes without playing with your pen or otherwise appearing to fidget.) In terms of specific things that should get your pen moving, among others, I would focus on specific words they use and concerns they have so that you can come back to them later in the conversation. Numbers are another good thing to hit the notebook, because numbers should only ever trend in a direction of “Better to you,” so you don’t want to do something stupid like saying “So how many days of vacation was that again?” and let a 24 suddenly become a 20.

Don’t say stupid things during job interviews such as “I need this job because…” You never need a job. Being needy means that the party who is not needy has automatic leverage over you. Instead of being needy, aim for “I’m enthusiastic about the opportunity with working with you, assuming we can come to mutually satisfactory terms.”

Micro-tip: Notice how often I say “We” and variations on “mutual win.” Those work pretty well. The only thing better than “We” is “You” (and variants), because people care a heck of a lot more about their problems than about your problems. This means that a) you should talk about their problems, concerns, and wishes and b) you should guard against your own natural tendency to bring up irrelevant things like your own problems, which typically will not help you sell the decisionmaker on adopting the mutual win you’re proposing. Similarly, I generally try to phrase things positively rather than score debating points. (“You just said X, but that was contradicted by your earlier statement Y, which means…” wins debating points but does not win friends and influence people. You might try something like “Good good, but taking into account your earlier concerns about Y…”)

You should, before any job interview, have intimate knowledge of the target company. Prospective peers within the company are one obvious way to get it. So are ex-employees, folks who’ve had dealings with them professionally, etc. Key things you want to learn:

  • What do they value?

  • Who do they value within the company? (Roles? Titles? Groups?)

  • What does the career path look like for successful people within the company?

  • Roughly speaking, how generous are they with regard to axes that you care about?

  • Do they have any compensation levers which are anomalously easy to operate? (For example, if you asked around, you might hear a few people say that a particular firm pushes back modestly on out-of-band increases in salary but they’ll give in-the-money option grants like candy.)

  • All the fuzzy stuff: what’s the corporate culture like? Yadda yadda.

You can even bring a lot of these questions to the job interview, which is (again) prior to the negotiation. (Maybe not “So are you guys tightwads?” but culture-esque questions like “What are the projects this company thinks are really key to its future and how would a motivated person go about getting on them?” are both a) totally fair game and b) will win you brownie points just for asking. Similarly, a lot of employees will, out of company loyalty, attempt to sell you on taking the job with the company by trading you very useful information.)

Case 5

Company: We can’t see our way to $88,000.

Applicant: Well, I know you do a significant amount of business with your online store. At my last company, I increased sales by 3% by $YADDA_YADDA. What would a 1% increase in sales be worth to you?

Company: Well, I don’t have that figure in front of me, but…

Applicant: Would it be safe to say “millions of dollars”?

Company: That sounds about right, yeah.

Applicant: Great, I can’t wait to get started. Getting me that extra $4,000 would make this a much easier decision. Considering that this is conceivably worth millions to you, we’d be silly not to do business with each other.

Case 6

Employer: “We were thinking $80,000.”

Applicant: “$80,000 is interesting (*) but not quite where we need to be to get this done. Do you have any flexibility on that number?”

Employer: “I think I can convince HR to approve $84,000 but that is the best I can do.”

Applicant: “I appreciate that. $84,000, huh. Well, it isn’t quite what I had in mind, but the right package offer could make that attractive. How much vacation comes with the package?”

Employer: “20 days a year.”

Applicant: “If you could do 24 days a year, I could compromise on $84,000.”

Employer: “I think I can do that.”

“Interesting” is a wonderful word: it is positive and non-commital at the same time. If they tell you a number, tell them it is an “interesting” number, not a “wonderful” number.

Hoping around the offer also helps you defuse common negotiating tactics like “I have to go to $EXTERNAL_AUTHORITY to get approval of that.” (This is in the negotiation playbook, because it works well: it injects an automatic delay in the process, and gives you a scapegoat for refusing a request while not being guilty of the refusal yourself. You should strongly consider having an $EXTERNAL_AUTHORITY of your own. Significant others work well. Note that in the US your would-be employer is legally prohibited from breathing about the subject of your marital status, so something like “We’ll have to talk that over” or “That sounds reasonable, but I’ll have to run it by the family” has the dual virtues of being a socially acceptable reason to delay any major decision.

Software engineering

Testing

Testing now will help prevent weird surprises later.

TDD:
  • Design/write the interface for your class

  • Write tests that exercise that interface…​ giving fixed input and expecting fixed output.

  • All tests will initially fail because the class was not written yet…​ you only have the interface.

  • Begin writing class implementation.

  • More and more tests gradually pass as the functionality gets added

  • Once all tests pass, you know the class is complete and functional (assuming you have tests which exercise all functionality)

Development-driven testing:
  • First, I think carefully about the specification, and write an implementation to handle every conceivable corner case. This usually compiles and works on the first try, except when I encounter compiler bugs (which is a frequent occurrence, working on the frontier of C++).

  • Then I go back to the specification, and write careful tests, trying to exercise every corner case. Generally I can’t be quite as exhaustive about this (the universe of possible inputs to template code is extremely infinite). This usually passes on the first try too, but sometimes reveals more compiler bugs, or occasionally some interaction with the language that I didn’t think of.

  • The important thing is not the order in which I write tests, but the deep thinking about the specification. The reverse order with equal thought would be okay, I just don’t work like that. What I am allergic to is the notion that a few test cases can be written up front, then anything that passes those tests is good. It is rarely possible to actually write tests that cover every conceivable boundary condition, whereas it is far easier to write code that handles them.

Debugging

Comment your code well. That doesn’t simply mean “Add a lot of com- ments.” You don’t say in English what is better said in code. Rather, you say in the comments — as clearly and briefl y as you can — what can’t be said clearly in code:

Practical debug advice:
  • The name of the program

  • The purpose of the program

  • Who wrote this code and when

  • Version numbers

  • What complicated code fragments are supposed to do

  • What the general design ideas are

  • How the source code is organized

  • What assumptions are made about inputs

  • What parts of the code are still missing and what cases are still not handled

  • Use meaningful names - That doesn’t simply mean “Use long names.”

  • Use a consistent layout of code.

  • Break code into small functions, each expressing a logical action. + Try to avoid functions longer than a page or two; most functions will be much shorter.

  • Avoid complicated code sequences. + Try to avoid nested loops, nested if -statements, complicated condi- tions, etc. Unfortunately, you sometimes need those, but remember that complicated code is where bugs can most easily hide.

  • Use library facilities rather than your own code when you can. + A library is likely to be better thought out and better tested than what you could produce as an alternative while busily solving your main problem.

Conducting Experiments
  • Testing systems with small examples to verify that they conform to the documentation or to understand their response when there is no documentation,

  • Testing small code changes to see if they actually fix a bug,

  • Measuring the performance of a system under two different conditions due to imperfect knowledge of their performance characteristics,

  • Checking the integrity of data, and

  • Collecting statistics that may hint at the solution to difficult or hard-to-repeat bugs.

Optimization
How to Optimize Loops
  • Remove floating point operations.

  • Don’t allocate new memory blocks unnecessarily.

  • Fold constants together.

  • Move I/O into a buffer.

  • Try not to divide.

  • Try not to do expensive typecasts.

  • Move a pointer rather than recomputing indices.

Dealing with I/O expense
  • caching (avoiding I/O)

  • representation (making I/O cheaper by representing data more efficiently)

  • locality (push the computation closer to the data.)

Classic Software Development Mistakes

If you’re not actively scanning through the list of Classic Software Development Mistakes as you run your software project, you have no idea how likely it is you’re making one of these mistakes right now.

People Mistakes
  • Undermined motivation

  • Weak personnel

  • Uncontrolled problem employees

  • Heroics

  • Adding people to a late project

  • Noisy, crowded offices

  • Friction between developers and customers

  • Unrealistic expectations

  • Lack of effective project sponsorship

  • Lack of stakeholder buy-in

  • Lack of user input

  • Politics placed over substance

  • Wishful thinking

Process Mistakes
  • Overly optimistic schedules

  • Insufficient risk management

  • Contractor failure

  • Insufficient planning

  • Abandonment of planning under pressure

  • Wasted time during the fuzzy front end

  • Shortchanged upstream activities

  • Inadequate design

  • Shortchanged quality assurance

  • Insufficient management controls

  • Premature or too frequent convergence

  • Omitting necessary tasks from estimates

  • Planning to catch up later

  • Code-like-hell programming

Product Mistakes
  • Requirements gold-plating

  • Feature creep

  • Developer gold-plating

  • Push me, pull me negotiation

  • Research-oriented development

Technology Mistakes
  • Silver-bullet syndrome

  • Overestimated savings from new tools or methods

  • Switching tools in the middle of a project

  • Lack of automated source control

Talks and conferences

  1. Meeting people is more important than the talks.

  2. Use shirt color to meet a diverse set of people.

  3. Don’t eat alone, eat with strangers not friends.

  4. Freely hand out your business card.

  5. Remember people you’ve met and follow up.

  6. Twitter is great for followup.

  7. The Goal of a Conference is to Meet People

If you haven’t met anyone (and followed up with them), you’re doing conferences wrong. Remember people’s names. Make the effort to remember them, and soon after the conversation, write it down somewhere. Add them to your social db.

Shirt Color Trick

Avoid people bias and talk to random people. A cool technique is shirt color trick. Pick a color besides black, grey, or blue and find the closest person wearing a shirt of that color. Go up and talk to them, no matter who they are. So if you pick green, find the closest person in a green shirt. If there’s nobody wearing a green shirt, pick a new color. Picking an uncommon color limits your choice of people to talk to, which limits how much your bias can influence you.

Ditch Your Friends

Talk to strangers. It’s comfortable to hang out with friends, but resist the temptation. When you show up for a lunch or dinner, find a table with people you know and sit at another table. If you are with your friends, try to include strangers who are standing/sitting nearby.

Don’t Eat Alone

Each at the conference’s lunches and talk to other people. Don’t sit with your friends, but join tables with people you don’t know. If you don’t have Sponsor booths in the expo halls will have fliers for after-party events as well.

Ditch the Talks

You don’t have to attend the talks. In fact, ditch them so you can meet people in the hallway. The talks will be posted online. If there is a speaker you want to meet, go to their talk, and sit front and center.

(Attend talks that you like, but don’t feel like you have to fill up your entire time with talks. And the keynotes are great to see in person rather than in a web browser.)

Business Cards

Get business cards printed, and bring a large amount of them with you. It’s better to have extras left over than run out halfway through the conference. They’re good parting gifts to exchange at the end of a conversation. If you are buying cards for yourself, order the minimum quantity of cards. Ordering a 1000 cards might be cheapest per card, but you’ll want to change the design or the information on them far before you finish handing them out.

Here’s what I have on my card: . My name, using the first name that I want people to call me by (Al, not Albert). . My email address. . My Twitter handle. (@frainfreeze) . My website.

Not my phone number. Not my address. (You should feel comfortable if someone posted your business card online.) You can write this info on the card if they need it. When you receive someone else’s card, on the back write the details of how you met them and what you talked about. Write “XCon {{date.year}}” on the card for when you find their card several months from now.

Twitter Account & Followup

Writing a direct email is time consuming and direct. But replying to a tweet they post is a great, low-commitment way to enter a conversation with them.

CLI

Tools

Touch typing
  • gnu typist

  • klava.org

  • typingclub.com

  • typing.io

  • speedcoder.net

Vim
Git
  • swcarpentry.github.io/git-novice/

  • git-scm.com/book/en/v2

  • documentup.com/skwp/git-workflows-book

  • eagain.net/articles/git-for-computer-scientists/

  • maryrosecook.com/blog/post/git-from-the-inside-out

Bash

Find a problem and try to solve it. Google 'how do i do x in bash'. Run your work through shellcheck. Repeat

Bash Basics
# single line comment

: '
A multiline
comment.
'

# shebang - allows script to be executed as executable
#!/usr/bin/env bash

# numeric calculations
x=4
y=7
echo $((x + 200))     # Add 200 to $x, notice that we dont use $ prefix in arithmetic expansions
echo $(( x + y ))     # 11
echo $(( ++x + y++ )) # 12
$((RANDOM%=200))  # Random number 0..200

# printing text, variables and strings
NAME="John"
printf "Hello %s, I'm %s" $NAME Tom # => Hello John, I'm Tom
echo $NAME      # => John
echo "$NAME"    # => John
echo "${NAME}!" # => John!
local local_var="I'm a local value"
export GLOBAL_VAR="I'm a global variable"

# reading input
echo -n "Proceed? [y/n]: "
read ans
echo $ans

read -n 1 ans    # Just one character

# conditional execution
git commit && git push
git commit || echo "Commit failed"

# Command substitution
now=`date +%T`
now=$(date +%T)
echo $now # 19:08:26

# Subshells
(cd somedir; echo "I'm now in $PWD")

# brace expansion
echo {A,B}         # => A B
echo {A,B}.js    # => A.js B.js
echo beg{i,a,u}n # begin began begun
echo {1..5}        # => 1 2 3 4 5
echo {00..8..2}  # 00 02 04 06 08

# variables/command substitutions dont expand inside single quotes
echo "Your home: $HOME" # => Your home: /Users/<username>
echo 'Your home: $HOME' # => Your home: $HOME

# expanding and quotes
INPUT="A string  with   strange    whitespace."
echo $INPUT   # A string with strange whitespace.
echo "$INPUT" # A string  with   strange    whitespace.

FILE="Favorite Things.txt"
cat $FILE   # attempts to print 2 files: `Favorite` and `Things.txt`
cat "$FILE" # prints 1 file: `Favorite Things.txt`
Parameter expansions
# Basics
name="John"
echo ${name}
echo ${name/J/j}    # => "john" (substitution)
echo ${name:0:2}    # => "Jo" (slicing)
echo ${name::2}     # => "Jo" (slicing)
echo ${name::-1}    # => "Joh" (slicing)
echo ${food:-Cake}  # => $food or "Cake"

length=2
echo ${name:0:length}  # => "Jo"

STR="/path/to/foo.cpp"
echo ${STR%.cpp}    # /path/to/foo
echo ${STR%.cpp}.o  # /path/to/foo.o
echo ${STR##*.}     # cpp (extension)
echo ${STR##*/}     # foo.cpp (basepath)
echo ${STR#*/}      # path/to/foo.cpp
echo ${STR##*/}     # foo.cpp
echo ${STR/foo/bar} # /path/to/bar.cpp

STR="Hello world"
echo ${STR:6:5}   # "world"
echo ${STR:-5:5}  # "world"

SRC="/path/to/foo.cpp"
BASE=${STR##*/}   #=> "foo.cpp" (basepath)
DIR=${SRC%$BASE}  #=> "/path/to" (dirpath)


# Substitution
${FOO%suffix}    # Remove suffix
${FOO#prefix}    # Remove prefix
${FOO%%suffix}   # Remove long suffix
${FOO##prefix}   # Remove long prefix
${FOO/from/to}   # Replace first match
${FOO//from/to}  # Replace all
${FOO/%from/to}  # Replace suffix
${FOO/#from/to}  # Replace prefix


# Substrings
${FOO:0:3}   # Substring (position, length)
${FOO:-3:3}  # Substring from the right


# Length
${#FOO}     # Length of $FOO


# Ternary conditions/Default values
# The : is optional (eg, ${FOO=word} works)
${FOO:-val}      # $FOO, or val if not set
${FOO:=val}      # Set $FOO to val if not set
${FOO:+val}      # val if $FOO is set
${FOO:?message}  # Show error message and exit if $FOO is not set
${var:ofs:len}   # substring expansion. returns substring of $var starting at ofs and up to len characters
Loops
# Basic for loop
for i in /etc/rc.*; do
  echo $i
done


# Infinite while loop
while true; do
  ···
done


# Until syntax
until condition; do
  statements
done


# select syntax
select answer in elem1 elem2 ... elemN
do
  # statements
done


# Ranges
for i in {1..5}; do
    echo "Welcome $i"
done


# with step size
for i in {5..50..5}; do
    echo "Welcome $i"
done


# Reading lines
cat file.txt | while read line; do
  echo $line
done

# Continue - next iteration, break - leave loop
for (( i = 0; i < 10; i++ )); do
  if [[ $(( i % 2 )) -eq 0 ]]; then continue; fi
  echo $i
done # => 1 3 5 7 9
# complex example
PS3="Choose the package manager: "
select ITEM in bower npm gem pip
do
  echo -n "Enter the package name: " && read PACKAGE
  case $ITEM in
    bower) bower install $PACKAGE ;;
    npm)   npm   install $PACKAGE ;;
    gem)   gem   install $PACKAGE ;;
    pip)   pip   install $PACKAGE ;;
  esac
  break # avoid infinite loop
done
$ ./my_script
1) bower
2) npm
3) gem
4) pip
Choose the package manager: 4
Enter the package name: ptpython
<installing ptpython>
Functions
# Defining functions: syntax 1
myfunc() {
    echo "hello $1"
}

# Defining functions: syntax 2
function myfunc() {
    echo "hello $1"
}

myfunc "John" # => hello John


# Returning values
myfunc() {
    local myresult='some value'
    echo $myresult
}

result=$(myfunc)
echo $result     # => some value


# Raising errors
myfunc() {
  return 1
}

if myfunc; then
  echo "success"
else
  echo "failure"
fi


# Arguments
$#  # Number of arguments
$*  # All arguments
$@  # All arguments, starting from first
$1  # First argument
$?  # Exit status of last task
$!  # PID of last background task
$$  # PID of shell
Conditionals
Conditions Desc

[ -z STRING ]

# Empty string

[ -n STRING ]

# Not empty string

[ NUM -eq NUM ]

# Equal

[ NUM -ne NUM ]

# Not equal

[ NUM -lt NUM ]

# Less than

[ NUM -le NUM ]

# Less than or equal

[ NUM -gt NUM ]

# Greater than

[ NUM -ge NUM ]

# Greater than or equal

[[ STRING =~ STRING ]]

# Regexp

NUM < NUM

# Numeric conditions

[ -o noclobber ]

# If OPTIONNAME is enabled

[ ! EXPR ]

# Not

[ X ] && [ Y ]

# And

[ X ]

[ Y ]

# Or

File conditions Desc

[ -e FILE ]

# Exists

[ -r FILE ]

# Readable

[ -h FILE ]

# Symlink

[ -d FILE ]

# Directory

[ -w FILE ]

# Writable

[ -s FILE ]

# Size is > 0 bytes

[ -f FILE ]

# File

[ -x FILE ]

# Executable

[ FILE1 -nt FILE2 ]

# 1 is more recent than 2

[ FILE1 -ot FILE2 ]

# 2 is more recent than 1

[ FILE1 -ef FILE2 ]

# Same files

### Examples ###
if (( $a < $b ))

if [ -e "file.txt" ]; then
  echo "file exists"
fi

# String
if [ -z "$string" ]; then
  echo "String is empty"
elif [ -n "$string" ]; then
  echo "String is not empty"
fi

# Combinations
if [ X ] && [ Y ]; then
  ...
fi

# Regex
if [[ "A" =~ "." ]]
Arrays
# Defining arrays
Fruits=('Apple' 'Banana' 'Orange')
Fruits[0]="Apple"
Fruits[1]="Banana"
Fruits[2]="Orange"


# Operations
Fruits=("${Fruits[@]}" "Watermelon")    # Push
Fruits=( ${Fruits[@]/Ap*/} )            # Remove by regex match
unset Fruits[2]                         # Remove one item
Fruits=("${Fruits[@]}")                 # Duplicate
Fruits=("${Fruits[@]}" "${Veggies[@]}") # Concatenate
lines=(`cat "logfile"`)                 # Read from file


# Working with arrays
echo ${Fruits[0]}      # Element #0
echo ${Fruits[@]}      # All elements, space-separated
echo ${#Fruits[@]}     # Number of elements
echo ${#Fruits}        # String length of the 1st element
echo ${#Fruits[3]}     # String length of the Nth element
echo ${#Fruits[@]}     # Num of values in array
echo ${Fruits[@]:3:2}  # Range (from position 3, length 2)


#Iteration
for i in "${arrayName[@]}"; do
  echo $i
done
Streams and pipes
Code Descriptor Description

0

stdin

The standard input.

1

stdout

The standard output.

2

stderr

The errors output.

Operator Description

>

Redirecting output

&>

Redirecting output and error output

&>>

Appending redirected output and error output

<

Redirecting input

<<

Here documents syntax

<<<

Here strings

# Redirection examples
python hello.py > output.txt   # stdout to (file)
python hello.py >> output.txt  # stdout to (file), append
python hello.py 2> error.log   # stderr to (file)
python hello.py 2>&1           # stderr to stdout
python hello.py 2>/dev/null    # stderr to (null)
python hello.py &>/dev/null    # stdout and stderr to (null)
python hello.py < foo.txt      # stdin to script (?)
Misc
# Inspecting commands
command -V cd # => "cd is a shell builtin"


# Debugging
bash -n scriptname
# is same as:
#!/bin/bash -n
# you can also turn options on and of with +-
set +x
...
-n # noexec     Read commands, but don't execute them (syntax check).
-v # verbose    Print each command to stderr before executing it.
-x # xtrace     Print each command and its expanded arguments to stderr before executing it.
-f # noglob     Disable filename expansion (globbing).
-i # interactive    Script runs in interactive mode.
-t # Exit after first command.


# Directory of script
DIR="${0%/*}"


# Switch
case "$1" in
  start | up)
    vagrant up
    ;;

  *)
    echo "Usage: $0 {start|stop|ssh}"
    ;;
esac


# Processing options
while [[ "$1" =~ ^- && ! "$1" == "--" ]]; do case $1 in
  -V | --version )
    echo $version
    exit
    ;;
  -s | --string )
    shift; string=$1
    ;;
  -f | --flag )
    flag=1
    ;;
esac; shift; done
if [[ "$1" == '--' ]]; then shift; fi


# Heredoc
cat <<END
hello world
END


### Options ###
set -o noclobber  # Avoid overlay files (echo "hi" > foo)
set -o errexit    # Used to exit upon error, avoiding cascading errors
set -o pipefail   # Unveils hidden failures
set -o nounset    # Exposes unset variables

# Glob options
set -o nullglob    # Non-matching globs are removed  ('*.foo' => '')
set -o failglob    # Non-matching globs throw errors
set -o nocaseglob  # Case insensitive globs
set -o globdots    # Wildcards match dotfiles ("*.sh" => ".foo.sh")
set -o globstar    # Allow ** for recursive matches ('lib/**/*.rb' => 'lib/a/b/c.rb')


### History ###
history             # Show history
shopt -s histverify # Don’t execute expanded result immediately

# Operations
# !! and !$ can be replaced with any valid expansion.
!!:s/<FROM>/<TO>/   # Replace first occurrence of <FROM> to <TO> in most recent command
!!:gs/<FROM>/<TO>/  # Replace all occurrences of <FROM> to <TO> in most recent command
!$:t                # Expand only basename from last parameter of most recent command
!$:h                # Expand only directory from last parameter of most recent command

# Expansions
!$      # Expand last parameter of most recent command
!*      # Expand all parameters of most recent command
!-n       # Expand nth most recent command
!n      # Expand nth command in history
!<cmd>  # Expand most recent invocation of command <cmd>

# Slices
# !! can be replaced with any valid expansion i.e. !cat, !-2, !42, etc.
!!:n    # Expand only nth token from most recent command (command is 0; first param is 1)
!!:n-m  # Expand range of tokens from most recent command
!!:n-$  # Expand nth token to last from most recent command

Linux commands

  • ab or wrk: benchmarking web servers

  • alias: allows launching of any command or combination of commands by using a preset character or series of characters.

  • apg: generates random passwords

  • apropos: displays a list of all topics in the built-in user manual that are related to the subject of a query.

  • bc: calculator

  • bzip2: used for compressing and decompressing files.

  • cal: nice calendar

  • cat: (short for concatenate) has three related functions with regard to text files: displaying them, combining copies of them and creating new ones.

  • cd: changes directories.

  • clear: removes all previous commands and output from consoles and terminal windows.

  • column: format text fields into aligned, fixed-width columns or tables

  • comm: compare sorted files line by line

  • cp: copies files and directories.

  • cssh: visual concurrent shell

  • cut, paste and join: data manipulation

  • dd: moving data between files or devices

  • df: reports the amount of space used and available on currently mounted filesystems.

  • dmesg: boot and system error messages

  • dmesg: reads the kernel messages.

  • dstat: useful system stats

  • du: shows the sizes of directories and files.

  • env: run a command (useful in scripts)

  • expand and unexpand: convert between tabs and spaces

  • expr: perform arithmetic or boolean operations or evaluate regular expressions

  • factor: factor integers

  • fdformat: performs low-level formatting of floppy disks.

  • file: classifies filesystem objects.

  • file: identify type of a file

  • fmt: format text paragraphs

  • fold: wrap lines of text

  • fortune and sl: fun but not ``useful''

  • free: provides information about unused and used memory and swap space.

  • glances: high level, multi-subsystem overview

  • gpg: encrypt and sign files

  • grep: searches text.

  • hdparm: SATA/ATA disk manipulation/performance

  • head: by default reads the first ten lines of text.

  • host and dig: DNS lookups

  • hostname: shows or sets a computer’s host name and domain name.

  • htop: improved version of top

  • iconv or uconv: conversion for text encodings

  • id: user/group identity info

  • iftop or nethogs: network utilization by socket or process

  • iostat: Disk usage stats

  • kdesu: opens KDE su, the graphical front end for the su command.

  • killall: terminates all processes associated with programs whose names are provided to it as arguments.

  • kill: terminates stalled processes without having to log out or reboot.

  • last: login history

  • ldd: dynamic library info

  • locate: finds files and directories.

  • lockfile: create semaphore file that can only be removed by rm -f

  • logrotate: rotate, compress and mail logs.

  • look: find English words (or lines in a file) beginning with a string

  • lsblk: list block devices:** a tree view of your disks and disk partitions

  • lshw, lscpu, lspci, lsusb, dmidecode: hardware information, including CPU, BIOS, RAID, graphics, devices, etc.

  • lsmod and modinfo: List and show details of kernel modules.

  • lsof: process file descriptor and socket info

  • m4: simple macro processor

  • man: formats and displays the built-in manual pages.

  • mkbootdisk: creates an emergency boot floppy.

  • mkdir: creates new directories.

  • mkfs: creates a filesystem on a disk or on a partition thereof.

  • mpstat: CPU usage stats

  • mtr: better traceroute for network debugging

  • mv: renames and moves files and directories.

  • nc: network debugging and data transfer

  • ngrep: grep for the network layer

  • nl: add line numbers

  • nm: symbols from object files

  • pr: format text into pages/columns

  • printenv: print out environment variables (useful in debugging and scripts)

  • ps: (short for process status) lists the currently running processes and their process identification numbers (PIDs).

  • pstree: displays the processes on the system in the form of a tree diagram.

  • pwd: (short for present working directory) displays the full path to the current directory.

  • reboot: restarts a computer without having to turn the power off and back on.

  • rm: deletes the specified files and directories.

  • rmdir: deletes the specified empty directories.

  • rsync: sync files and folders over SSH or in local file system

  • runlevel: reports the current and previous runlevels.

  • sar: historic system stats

  • seq: print numbers

  • shred: destroys files.

  • slurm: network traffic visualization

  • socat: socket relay and tcp port forwarder (similar to netcat)

  • spell: checks spelling.

  • split and csplit: splitting files

  • sponge: read all input before writing it, useful for reading from then writing to the same file, e.g., grep -v something some-file | sponge some-file

  • ss: socket statistics

  • stat: file info

  • strace: system call debugging

  • strings: extract text from binary files

  • strings: returns each string of printable characters in files.

  • su: (short for substitute user) changes a login session’s owner without the owner having to first log out of that session.

  • sysctl: view and configure Linux kernel parameters at run time

  • tac: print files in reverse

  • tail: by default reads the final ten lines of text.

  • tar: converts a group of files into an archive.

  • time: execute and time a command

  • timeout: execute a command for specified amount of time and stop the process when the specified amount of time completes.

  • toe: table of terminfo entries

  • touch: the easiest way to create new, empty files.

  • tr: character translation or manipulation

  • tree: display directories and subdirectories as a nesting tree; like ls but recursive

  • tr: translates or deletes characters.

  • unalias: removes entries from the current user’s list of aliases.

  • uname: provides basic information about a system’s software and hardware.

  • units: unit conversions and calculations; converts furlongs per fortnight to twips per blink (see also /usr/share/units/definitions.units)

  • uptime: shows the current time, how long the system has been running since it was booted up, how many user sessions are currently open and the load averages.

  • vmstat: Memory usage stats

  • watch: run a command repeatedly, showing results and/or highlighting changes

  • wc: by default counts the number of lines, words and characters that are contained in text.

  • whatis: provides very brief descriptions of command line programs and other topics related to Unix-like operating systems.

  • when-changed: runs any command you specify whenever it sees file changed. See inotifywait and entr as well.

  • whereis: locates the binary, source code and man page for any specified program.

  • whoami: returns the user name of the owner of the current login session.

  • wireshark and tshark: packet capture and network debugging

  • w: shows who is logged into the system and what they are doing.

  • w: who’s logged on

  • xz: high-ratio file compression

  • yes: print a string a lot

Git — Fast Version Control

Directory and files

Repository

A set of files, directories, historical records, commits, and heads. Imagine it as a source code data structure, with the attribute that each source code “element” gives you access to its revision history, among other things.

A git repository is comprised of the .git directory & working tree.

.git Directory (component of repository)

The .git directory contains all the configurations, logs, branches, HEAD, and more. Basic structure looks something like this:

.
|-- COMMIT_EDITMSG
|-- FETCH_HEAD
|-- HEAD
|-- ORIG_HEAD
|-- branches
|-- config
|-- description
|-- hooks
|   |-- applypatch-msg
|   |-- commit-msg
|   |-- post-commit
|   |-- post-receive
|   |-- post-update
|   |-- pre-applypatch
|   |-- pre-commit
|   |-- pre-rebase
|   |-- prepare-commit-msg
|   `-- update
|-- index
|-- info
|   `-- exclude
|-- logs
|   |-- HEAD
|   `-- refs
|-- objects
`-- refs
    |-- heads
    |-- remotes
    |-- stash
    `-- tags

Let’s go over some of the normal files that you may see living in the base directory:

  • COMMIT_EDITMSG: This is the last commit’s message. It’s not actually used by Git at all, but it’s there mostly for your reference after you made a commit.

  • config: Contains settings for this repository. Specific configuration variables can be dumped in here (and even aliases!) What this file is most used for is defining where remotes live and some core settings, such as if your repository is bare or not.

  • description: If you’re using gitweb or firing up git instaweb, this will show up when you view your repository or the list of all versioned repositories.

  • FETCH_HEAD: The SHAs of branch/remote heads that were updated during the last git fetch

  • HEAD: The current ref that you’re looking at. In most cases it’s probably refs/heads/master

  • index: The staging area with meta-data such as timestamps, file names and also SHAs of the files that are already wrapped up by Git.

  • packed-refs: Packs away dormant refs, this is not the definitive list of refs in your repository (the refs folder has the real ones!) Take a look at gitster’s comment to see more information on this.

  • ORIG_HEAD: When doing a merge, this is the SHA of the branch you’re merging into.

  • MERGE_HEAD: When doing a merge, this is the SHA of the branch you’re merging from.

  • MERGE_MODE: Used to communicate constraints that were originally given to git merge to git commit when a merge conflicts, and a separate git commit is needed to conclude it. Currently --no-ff is the only constraints passed this way.

  • MERGE_MSG: Enumerates conflicts that happen during your current merge.

  • RENAMED-REF: Still trying to track this one down. From a basic grep through the source, it seems like this file is related to errors when saving refs.

There’s plenty of directories as well:

  • hooks: A directory that will fast become your best friend: this contains scripts that are executed at certain times when working with Git, such as after a commit or before a rebase. An entire series of articles will be coming about hooks.

  • info: Relatively uninteresting except for the exclude file that lives inside of it. We’ve seen this before in the ignoring files article, but as a reminder, you can use this file to ignore files for this project, but beware! It’s not versioned like a .gitignore file would be.

  • logs: Contains history for different branches. Seems to be used mostly with the reflog command.

  • objects: Git’s internal warehouse of blobs, all indexed by SHAs.

  • rebase-apply: The workbench for rebasing and for git am. You can dig into its patch file when it does not apply cleanly if you’re brave.

  • refs: The master copy of all refs that live in your repository, be they for stashes, tags, remote tracking branches, or local branches.

Working Tree (component of repository)

This is basically the directories and files in your repository. It is often referred to as your working directory.

Index (component of .git dir)

The Index is the staging area in git. It’s basically a layer that separates your working tree from the Git repository. This gives developers more power over what gets sent to the Git repository.

Commit

A git commit is a snapshot of a set of changes, or manipulations to your Working Tree. For example, if you added 5 files, and removed 2 others, these changes will be contained in a commit (or snapshot). This commit can then be pushed to other repositories, or not!

Branch

A branch is essentially a pointer to the last commit you made. As you go on committing, this pointer will automatically update to point the latest commit.

Tag

A tag is a mark on specific point in history. Typically people use this functionality to mark release points (v1.0, and so on)

HEAD and head (component of .git dir)

HEAD is a pointer that points to the current branch. A repository only has 1 active HEAD. head is a pointer that points to any commit. A repository can have any number of heads.

Stages of Git

+ Modified - Changes have been made to a file but file has not been committed to Git Database yet + Staged - Marks a modified file to go into your next commit snapshot + Committed - Files have been committed to the Git Database

Commands

help
# Quickly check available commands
$ git help

# Check all available commands
$ git help -a

# Command specific help - user manual
# git help <command_here>
$ git help add
$ git help commit
$ git help init
# or git <command_here> --help
$ git add --help
$ git commit --help
$ git init --help
config

To configure settings. Whether it be for the repository, the system itself, or global configurations (global config file is ~/.gitconfig).

# Some Basic Config Variables (Global)
$ git config --global user.email "email@example.com"
$ git config --global user.name "nick"

# Print Config Variables
$ git config --global user.name
init

Create an empty Git repository. The Git repository’s settings, stored information, and more is stored in a directory (a folder) named “.git”.

$ git init
ignore files

To intentionally untrack file(s) & folder(s) from git. Typically meant for private & temp files which would otherwise be shared in the repository.

$ echo "temp/" >> .gitignore
$ echo "private_key" >> .gitignore
status

To show differences between the index file (basically your working copy/repo) and the current HEAD commit.

# Will display the branch, untracked files, changes and other differences
$ git status
add

To add files to the staging area/index. If you do not git add new files to the staging area/index, they will not be included in commits!

# add a file in your current working directory
$ git add HelloWorld.py

# add a file in a nested dir
$ git add /path/to/file/HelloWorld.c

# Regular Expression support!
$ git add ./*.cpp

# You can also add everything in your working directory to the staging area.
$ git add -A

This only adds a file to the staging area/index, it doesn’t commit it to the working directory/repo.

branch

Manage your branches. You can view, edit, create, delete branches using this command.

# list existing branches & remotes
$ git branch -a

# create a new branch
$ git branch myNewBranch

# delete a branch
$ git branch -d myBranch

# rename a branch
# git branch -m <oldname> <newname>
$ git branch -m myBranchName myNewBranchName

# edit a branch's description
$ git branch myBranchName --edit-description
tag

Manage your tags

# List tags
$ git tag

# Create a annotated tag
# The -m specifies a tagging message, which is stored with the tag.
# If you don’t specify a message for an annotated tag,
# Git launches your editor so you can type it in.
$ git tag -a v2.0 -m 'my version 2.0'

# Show info about tag
# That shows the tagger information, the date the commit was tagged,
# and the annotation message before showing the commit information.
$ git show v2.0

# Push a single tag to remote
$ git push origin v2.0

# Push a lot of tags to remote
$ git push origin --tags
checkout

Updates all files in the working tree to match the version in the index, or specified tree.

# Checkout a repo - defaults to master branch
$ git checkout

# Checkout a specified branch
$ git checkout branchName

# Create a new branch & switch to it
# equivalent to "git branch <name>; git checkout <name>"

$ git checkout -b newBranch
clone

Clones, or copies, an existing repository into a new directory. It also adds remote-tracking branches for each branch in the cloned repo, which allows you to push to a remote branch.

# Clone test
$ git clone https://github.com/frainfreeze/test.git

# shallow clone - faster cloning that pulls only latest snapshot
$ git clone --depth 1 https://github.com/frainfreeze/test.git

# clone only a specific branch
$ git clone -b master-cn https://github.com/frainfreeze/test.git --single-branch
commit

Stores the current contents of the index in a new “commit.” This commit contains the changes made and a message created by the user.

# commit with a message
$ git commit -m "Added multiplyNumbers() function to HelloWorld.c"

# signed commit with a message (user.signingkey must have been set
# with your GPG key e.g. git config --global user.signingkey 5173AAD5)
$ git commit -S -m "signed commit message"

# automatically stage modified or deleted files, except new files, and then commit
$ git commit -a -m "Modified foo.py and removed bar.py"

# change last commit (this deletes previous commit with a fresh commit)
$ git commit --amend -m "Correct message"
diff

Shows differences between a file in the working directory, index and commits.

# Show difference between your working dir and the index
$ git diff

# Show differences between the index and the most recent commit.
$ git diff --cached

# Show differences between your working dir and the most recent commit
$ git diff HEAD
grep

Allows you to quickly search a repository.

Optional Configurations:

# Thanks to Travis Jeffery for these
# Set line numbers to be shown in grep search results
$ git config --global grep.lineNumber true

# Make search results more readable, including grouping
$ git config --global alias.g "grep --break --heading --line-number"

# Search for "variableName" in all java files
$ git grep 'variableName' -- '*.java'

# Search for a line that contains "arrayListName" and, "add" or "remove"
$ git grep -e 'arrayListName' --and \( -e add -e remove \)
log

Display commits to the repository.

# Show all commits
$ git log

# Show only commit message & ref
$ git log --oneline

# Show merge commits only
$ git log --merges

# Show all commits represented by an ASCII graph
$ git log --graph
merge

“Merge” in changes from external commits into the current branch.

# Merge the specified branch into the current.
$ git merge branchName

# Always generate a merge commit when merging
$ git merge --no-ff branchName
mv

Rename or move a file

# Renaming a file
$ git mv HelloWorld.c HelloNewWorld.c

# Moving a file
$ git mv HelloWorld.c ./new/path/HelloWorld.c

# Force rename or move
# "existingFile" already exists in the directory, will be overwritten
$ git mv -f myFile existingFile
pull

Pulls from a repository and merges it with another branch.

# Update your local repo, by merging in new changes
# from the remote "origin" and "master" branch.
# git pull <remote> <branch>
$ git pull origin master

# By default, git pull will update your current branch
# by merging in new changes from its remote-tracking branch
$ git pull

# Merge in changes from remote branch and rebase
# branch commits onto your local repo, like: "git fetch <remote> <branch>, git
# rebase <remote>/<branch>"
$ git pull origin master --rebase
push

Push and merge changes from a branch to a remote & branch.

# Push and merge changes from a local repo to a
# remote named "origin" and "master" branch.
# git push <remote> <branch>
$ git push origin master

# By default, git push will push and merge changes from
# the current branch to its remote-tracking branch
$ git push

# To link up current local branch with a remote branch, add -u flag:
$ git push -u origin master
# Now, anytime you want to push from that same local branch, use shortcut:
$ git push
stash

Stashing takes the dirty state of your working directory and saves it on a stack of unfinished changes that you can reapply at any time.

Let’s say you’ve been doing some work in your git repo, but you want to pull from the remote. Since you have dirty (uncommitted) changes to some files, you are not able to run git pull. Instead, you can run git stash to save your changes onto a stack!

$ git stash
Saved working directory and index state \
  "WIP on master: 049d078 added the index file"
  HEAD is now at 049d078 added the index file
  (To restore them type "git stash apply")

Now you can pull!

git pull

Now check that everything is OK

$ git status
On branch master
nothing to commit, working directory clean
```bash
You can see what “hunks” you’ve stashed so far using git stash list. Since the “hunks” are stored in a Last-In-First-Out stack, our most recent change will be at top.
```bash
$ git stash list
stash@{0}: WIP on master: 049d078 added the index file
stash@{1}: WIP on master: c264051 Revert "added file_size"
stash@{2}: WIP on master: 21d80a5 added number to log

Now let’s apply our dirty changes back by popping them off the stack.

$ git stash pop
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#      modified:   index.html
#      modified:   lib/simplegit.py

git stash apply does the same thing

rebase (caution)

Take all changes that were committed on one branch, and replay them onto another branch. Do not rebase commits that you have pushed to a public repo.

# Rebase experimentBranch onto master
# git rebase <basebranch> <topicbranch>
$ git rebase master experimentBranch
reset (caution)

Reset the current HEAD to the specified state. This allows you to undo merges, pulls, commits, adds, and more. It’s a great command but also dangerous if you don’t know what you are doing.

# Reset the staging area, to match the latest commit (leaves dir unchanged)
$ git reset

# Reset the staging area, to match the latest commit, and overwrite working dir
$ git reset --hard

# Moves the current branch tip to the specified commit (leaves dir unchanged)
# all changes still exist in the directory.
$ git reset 31f2bb1

# Moves the current branch tip backward to the specified commit
# and makes the working dir match (deletes uncommitted changes and all commits
# after the specified commit).
$ git reset --hard 31f2bb1
reflog (caution)

Reflog will list most of the git commands you have done for a given time period, default 90 days.

This give you the chance to reverse any git commands that have gone wrong (for instance, if a rebase has broken your application).

You can run

$ git reflog
38b323f HEAD@{0}: rebase -i (finish): returning to refs/heads/feature/add_git_reflog
38b323f HEAD@{1}: rebase -i (pick): Clarify inc/dec operators
4fff859 HEAD@{2}: rebase -i (pick): Update java.html.markdown
34ed963 HEAD@{3}: rebase -i (pick): [yaml/en] Add more resources (#1666)
ed8ddf2 HEAD@{4}: rebase -i (pick): pythonstatcomp spanish translation (#1748)
2e6c386 HEAD@{5}: rebase -i (start): checkout 02fb96d

to list all of the git commands for the rebase 1. Select where to reset to, in our case its 2e6c386, or HEAD@{5} 2. git reset –hard HEAD@{5} this will reset your repo to that head 3. You can start the rebase again or leave it alone.

revert

Revert can be used to undo a commit. It should not be confused with reset which restores the state of a project to a previous point. Revert will add a new commit which is the inverse of the specified commit, thus reverting it.

# Revert a specified commit
$ git revert <commit>
rm

The opposite of git add, git rm removes files from the current working tree.

# remove HelloWorld.c
$ git rm HelloWorld.c

# Remove a file from a nested dir
$ git rm /pather/to/the/file/HelloWorld.c

Complete Solutions

Merging 2 git repositories without breaking file history

Add the second repo as a remote:

    cd firstgitrepo/
    git remote add secondrepo username@servername:andsoon

Make sure that you’ve downloaded all of the secondrepo’s commits:

    git fetch secondrepo

Create a local branch from the second repo’s branch:

    git branch branchfromsecondrepo secondrepo/master

Move all its files into a subdirectory:

    git checkout branchfromsecondrepo
    mkdir subdir/
    git ls-tree -z --name-only HEAD | xargs -0 -I {} git mv {} subdir/
    git commit -m "Moved files to subdir/"

Merge the second branch into the first repo’s master branch:

    git checkout master
    git merge --allow-unrelated-histories branchfromsecondrepo

Your repository will have more than one root commit, but that shouldn’t pose a problem

C++ reference and notes

Syntax

#include <iostream> // contains cout, endl
#include <cstdlib> // for random numbers
#include <ctime> // to seed rnd num generator
#include <string>
#include <vector>
#include <algorithm> // STL algorithms
/* libraries with c prefix are from c library
and can be also written as <name.h>, for e.g.
<ctime> is same as <time.h> */

using namespace std;


// global variables are defined outside functions
// const type - expression with fixed value
const double PI = 3.14; // global constant value


// function prototypes //
/* if no argument given b will be set to 5, once
argument is specified we have to specify others
after it as well */
int sumNums(int a, int b = 5);

inline void printX(); // inline function

void swapNums(int& a, int& b);


// function definitions //
void printZ(int a){
    cout << a+1 << endl;
}

void printZ(string a){
    cout << a+" bar" << endl;
}

string& ref2element(vector<string>& barVec, int i){
    return barVec[i]; // returns reference
}

void pointerExample(int* pointy){
    cout << *pointy << endl;
}

void pointerExample(const int* pointy){
    cout << *pointy << endl;
}

//returns a pointer to a string element
string* ptrToElement(vector<string>* const pVec, int i);

int main() {
    /*
    expressions
     ! | NOT
    && | AND
    || | OR

    precedence levels
    ! > && > ||

    * deference operator
    ++ increment operator
    [] subscripting operator
    */


    // declaring variables //
    char word[] = "word"; // cstyle string

    // unsigned type - has 2x value but no negatives
    // int -1 to -128, 0 to 127, unsigned int 0 to 255
    unsigned int unsignedintA = 1212121;


    // casting //
    // static_cast<type>(value) - converting (casting) value to type
    int intA = 10;
    static_cast<unsigned int>(intA); // converts intA into unsigned int


    // enumeration - set of unsigned int constants (enumerators) //
    // if you assign value to enumerator next one will have +1 value
    enum fields { WORD, HINT, NUM_FIELDS, example1 = 23, example2 };

    // currentField is of type field and has value 2
    fields currentField = HINT;

    // has value 24 (its 1+ of previous enumerator)
    fields example2a = example2;

    cout << currentField << " " << example2a << endl;


    // conditionals //
    /*
    ==  true if left operand is equal to right operand
    !=  true if left operand is not equal to right operand
    >   true if left operand is greater than right operand
    <   true if left operand is less than right operand
    >=  true if left operand is greater than or equal than right operand
    <=  true if left operand is less than or equal than right operand

    0 is false, 1 is true
    */
    cout << (0 == false) << endl; // true, outputs 1
    cout << (1 == true) << endl; // true, outputs 1


    //// Statements and flow control ////
    // Selection statements: if; else if; else, switch //
    if (true){
       cout << "True." << endl;
    } else if (true){
        // statement
    } else {
        // if everything else fails
    }

    //switch
    char choice = 'a';
    switch(choice){
        case 'a':
            cout << "Statement." << endl;
            break; // end switch, otherwise continues onto next case
        default:
            //gets executed if choice was not found
            break;
    }


    // Iteration statements: while, do while, for //
    //infinite loops
    for(;;){
        int i = 1; ++i;
        if (i >= 2){
            break; // exits loop
        }
        cout << "Infinite for loop." << endl;
        continue; // goes into next loop iteration...
        cout << "Bamboozled."; // ...hence this never gets executed
    }

    while (true){
        cout << "Infinite while loop." << endl;
        break;
    }

    do{
        cout << "Infinite while loop." << endl;
        break;
    }while(true);

    // for in for
    // inner for gets executed in full for every outer for iteration
    for(int i = 0; i<1; i++){
        for(int j = 0; j<2; j++){
            cout << "In" << endl;
        }
        cout << "Out" << endl;
    }


    // random numbers //
    // seeding random number generator
    srand(static_cast<unsigned int>(time(0)));

    int randNum = rand(); // random number between 0 and RAND_MAX
    int randNums = (randNum%6) + 1; // random number between 1 and 6


    // strings //
    string str1 = "!!!!";
    string str2 = ("!!!!");
    string str3 = (4, "!");
    string str4;
    string str5 = "Very long string.";
    string::npos; // element that is not in the string/container

    /*
    .length()/.size() return number of chars in string object (unsigned int)
    .find() - checks if literal is contained in string, returns 1st occurrence
    .erase() - removes substring from string object
    .empty() - returns bool value, predicate functions return bool values
    */

    // returns true
    cout << ((str1.size()==str1.length()) && (str1.size()==4)) << endl;

    // searches for "long", starting at 3rd position
    cout << str5.find("long", 3) << endl;

    // deletes everything from position 1 onwards
    str1.erase(1);

    // deletes 8 chars starting at 9th position
    str5.erase(9,8);

    cout << str5 + str1 << endl;

    // returns true
    cout << (str4.empty() == true) << endl;

    if(str5.find("long") != string::npos)
        cout << "Phrase is in string." << endl;


    // arrays //
    const int MAX_WORDS = 10;

    // spam1.size() == 10, [0] == foo, [1] == bar
    string spam1[MAX_WORDS] = {"foo", "bar"};

    // spam2.size() == 2, [0] == foo, [1] == bar
    string spam2[] = {"foo", "bar"};

    spam1[3] = "spaz";
    cout << spam1[3].size() << endl; // 4

    // multidimensional array
    const int ROWS = 3;
    const int COLUMNS = 3;
    char board[ROWS][COLUMNS] = { {'O', 'X', 'O'},
                                  {'O', 'Z', 'X'},
                                  {'X', 'O', 'O'} };

    cout << "In the center of board is " << board[1][1] << endl;

    // printing multidimensional array
    for (int x = 0; x<3; x++){
        for (int y = 0; y<3; y++){
            cout << board[x][y];
        }
        cout << endl;
    }


    // iterators, vectors and algorithms //
    /* containers - objects that let you store and access collection of
                    elements of same type
       iterators - objects that identify elements in containers and can
                   be manipulated to move among the elements
       vector - sequential container
       sequential container - retrieves values in sequence
    */

    // vector filled with 10 "spam" strings
    vector<string> vec(10,"spam");

    // copy of vector vec
    vector<string> vecCopy(vec);

    vec.push_back("BAR");

    // add new element to the end of the vector
    vec.push_back("element 12");

    // remove last element from vector
    vec.pop_back();

    // .size() - vector size
    cout << "vec size: " <<vec.size() << endl;

    /* iterators are like post it notes that
    we stick on elements in container. they
    are way to refer to element and we can
    change or access elements through them */

    // declares iterator for vector that contains string objects
    vector<string>::iterator myIterator;

    // constant iterators - read only access
    vector<string>::const_iterator iter;

    // .begin() - first vector element
    vector<string>::const_iterator vecFirst = vec.begin();

    // .end() - one empty element after last element
    vector<string>::const_iterator vecLast = vec.end();

    // make iterator element on position 0
    myIterator = vec.begin();

    // change vec[0] to "FOO" using that iterator
    *myIterator = "FOO";

    // myIterator->size() is syntax sugar for (*myIterator).size()
    cout << "Vec[0] size: " << myIterator->size() << endl;

    // .insert() - inserts element in front of element referenced by iterator
    // example inserts "spam" on position 0, moves all other elements once place up

    // returns iterator referencing newly inserted element(?)
    vec.insert(vec.begin(), "spam");

    // .erase() - removes element from container(vector)
    vec.erase(myIterator);

    // random_shuffle(range) - randomly shuffles containers elements
    random_shuffle(vec.begin(),vec.end());

    // sort(range) - sorts item in ascending order
    sort(vec.begin(),vec.end());

    // print whole vector using iterators
    for (iter = vecFirst; iter != vecLast; ++iter){
        cout << *iter << " ";
    } cout << endl;

    // remove all elements and sets vector size to 0
    vec.clear();

    // .empty() - returns bool whether vector is empty
    cout << vec.empty() << endl;

    // toupper(letter) - uppercase letter

    /* capacity() vector member function returns the
    capacity of a vector. in other words, the number
    of elements that a vector can hold before a program
    must reallocate more memory for it. */
    cout << "Vec size is: " << vec.size()
         << " Vec capacity is: " << vec.capacity() << endl;

    /* reserve() member function increases the capacity
    of a vector to the number supplied as an argument,
    that number has to be bigger than current vec size */
    vec.reserve(21);

    cout << "Vec capacity is: " << vec.capacity() << endl;


    // scopes //
    int intC = 10;
    {
        // scope nested inside main scope
        int intC = 30; // we hided original intC by defining
        intC = 25; // actually changing intC, still, only inside the scope
    }
    cout << intC << endl;


    // functions //
    cout << sumNums(4) << endl; // 9

    // (inline) function gets inserted inhere by compiler
    // faster execution time for smaller functions
    printX();

    //overloading functions
    printZ(3);
    printZ("foo");


    // references //
    // nicknames for variables

    int intD = 10;

    // if we want to prevent reference modifying we use const
    const int& rIntD = intD;

    cout << rIntD << endl;

    int intE = 2; int intF = 3;
    swapNums(intE, intF);
    cout << intE << " " << intF << endl; // 3 2

    // small example of reference to reference
    vector<string> barVec;
    barVec.push_back("foo");
    barVec.push_back("bar");

    // assignees returned reference to reference
    string& rStr = ref2element(barVec,1);
    cout << rStr << endl; // bar
    rStr = "spaz"; // rStr == barVec[1]
    cout << barVec[1] << endl; // spaz


    // pointers //
    // a pointer is a variable that can contain a memory address
    int zz = 1;
    int zz2 = 343;
    string szz;

    //declare a pointer
    int* pAPointer;

    //declare and initialize a pointer
    string * pTestpointer = nullptr;

    // assign address of variable/object to the pointer
    pAPointer = &zz;
    pTestpointer = &szz;

    // reassign pointer to new pointee
    pAPointer = &zz2;

    // modify object pointer points too (pointee) by deferencing
    *pTestpointer = "test string\n";

    // print pointer address then object it references
    cout << pAPointer << " | " << *pAPointer << endl;
    cout << pTestpointer << " | " << *pTestpointer;

    /* constant pointers can only point to objects
    they were initialized to point to. like all constants
    they have to be initialized on declaration */
    int* const pConstatnPointer = &zz;

    // pointers to a constant cant be used to change
    // value they point to.
    const int* pZz;

    /* constant pointer to a constant combines the
    restrictions of a constant pointer and a pointer
    to a constant. This means that a constant pointer
    to a constant can only point to the object that it
    was initialized to point to. In addition, it can’t
    be used to change the value of the object to which
    it points*/
    const int* const pCPC = &zz2;

    // when passing pointers to function
    // we have to use address of object
    pointerExample(&zz);

    // ...which is same as using pointer
    pointerExample(pCPC);

    // returning pointers
    vector<string> tempVec(10,"spam");
    cout << *( ptrToElement(&tempVec, 3) ) << endl;

    // Assigning a Returned Pointer to a Pointer
    string* pMagic = ptrToElement(&tempVec, 3);
    *pMagic = "magic";
    cout << tempVec[3] << endl;

    // an array name is a constant pointer to
    // the first element of the array
    string tempArray[3] = {"foo", "bar", "spam"};
    *tempArray="hehexd";
    cout << tempArray[0] << endl;


    // classes and dynamic memory //


    // inheritance and polymorphism //


    return 0;
}


// function definitions //
int sumNums(int a, int b ){
    /* we don't respecify arguments in function
       definition if we did it in prototype */
    return a + b;
}

inline void printX(){
    cout << "Inline function." << endl;
}

void swapNums(int& a, int& b){
    int temp = a;
    a = b;
    b = temp;
}

string* ptrToElement(vector<string>* const pVec, int i){
    //returns address of the string in position
    // i of vector that pVec points to
    return &((*pVec)[i]);
}

Terms

  • argument: Value passed to a function.

  • assignment: Obliterates an object’s current value and replaces that value by a new one.

  • block: Sequence of zero or more statements enclosed in curly braces.

  • buffer: A region of storage used to hold data. IO facilities often store input (or output) in a buffer and read or write the buffer independently from actions in the program. Output buffers can be explicitly flushed to force the buffer to be written. By default, reading cin flushes cout; cout is also flushed when the program ends normally.

  • built-in type Type, such as int, defined by the language.

  • cerr: ostream object tied to the standard error, which often writes to the same device as the standard output. By default, writes to cerr are not buffered. Usually used for error messages or other output that is not part of the normal logic of the program. character string literal Another term for string literal.

  • cin: istream object used to read from the standard input.

  • class: Facility for defining our own data structures together with associated operations. The class is one of the most fundamental features in C++. Library types, such as istream and ostream, are classes.

  • class: type A type defined by a class. The name of the type is the class name.

  • clog: ostream object tied to the standard error. By default, writes to clog are buffered. Usually used to report information about program execution to a log file.

  • comments: Program text that is ignored by the compiler. C++ has two kinds of comments: single-line and paired. Single-line comments start with a //. Everything from the // to the end of the line is a comment. Paired comments begin with a / and include all text up to the next /.

  • condition: An expression that is evaluated as true or false. A value of zero is false; any other value yields true.

  • cout: ostream object used to write to the standard output. Ordinarily used to write the output of a program.

  • curly: brace Curly braces delimit blocks. An open curly ({) starts a block; a close curly (}) ends one.

  • data structure: A logical grouping of data and operations on that data.

  • edit-compile-debug: The process of getting a program to execute properly.

  • end-of-file: System-specific marker that indicates that there is no more input in a file.

  • expression: The smallest unit of computation. An expression consists of one or more operands and usually one or more operators. Expressions are evaluated to produce a result. For example, assuming i and j are int`s, then `i + j is an expression and yields the sum of the two int values.

  • for statement: Iteration statement that provides iterative execution. Often used to repeat a calculation a fixed number of times.

  • function: Named unit of computation.

  • function body: Block that defines the actions performed by a function.

  • function name: Name by which a function is known and can be called.

  • header: Mechanism whereby the definitions of a class or other names are made available to multiple programs. A program uses a header through a #include directive.

  • if statement: Conditional execution based on the value of a specified condition. If the condition is true, the if body is executed. If not, the else body is executed if there is one.

  • initialize: Give an object a value at the same time that it is created.

  • iostream: Header that provides the library types for stream-oriented input and output.

  • istream: Library type providing stream-oriented input. library type Type, such as istream, defined by the standard library.

  • main: Function called by the operating system to execute a C++ program. Each program must have one and only one function named main.

  • manipulator: Object, such as std::endl, that when read or written “manipulates” the stream itself.

  • member function: Operation defined by a class. Member functions ordinarily are called to operate on a specific object.

  • method: Synonym for member function.

  • namespace: Mechanism for putting names defined by a library into a single place. Namespaces help avoid inadvertent name clashes. The names defined by the C++ library are in the namespace std.

  • ostream: Library type providing stream-oriented output.

  • parameter list: Part of the definition of a function. Possibly empty list that specifies what arguments can be used to call the function.

  • return type: Type of the value returned by a function.

  • source file: Term used to describe a file that contains a C++ program.

  • standard error: Output stream used for error reporting. Ordinarily, the standard output and the standard error are tied to the window in which the program is executed.

  • standard input: Input stream usually associated with the window in which the program executes.

  • standard library: Collection of types and functions that every C compiler must support. The library provides the types that support IO. C programmers tend to talk about “the library,” meaning the entire standard library. They also tend to refer to particular parts of the library by referring to a library type, such as the “iostream library,” meaning the part of the standard library that defines the IO classes.

  • standard output: Output stream usually associated with the window in which the program executes.

  • statement: A part of a program that specifies an action to take place when the program is executed. An expression followed by a semicolon is a statement; other kinds of statements include blocks and if, for, and while statements, all of which contain other statements within themselves.

  • std: Name of the namespace used by the standard library. std::cout indicates that we’re using the name cout defined in the std namespace.

  • string literal: Sequence of zero or more characters enclosed in double quotes ("a string literal").

  • uninitialized variable: Variable that is not given an initial value. Variables of class type for which no initial value is specified are initialized as specified by the class definition. Variables of built-in type defined inside a function are uninitialized unless explicitly initialized. It is an error to try to use the value of an uninitialized variable. Uninitialized variables are a rich source of bugs .

  • variable: A named object.

  • while statement: Iteration statement that provides iterative execution so long as a specified condition is true. The body is executed zero or more times, depending on the truth value of the condition.

  • () operator: Call operator. A pair of parentheses “()” following a function name. The operator causes a function to be invoked. Arguments to the function may be passed inside the parentheses.

  • ** operator**: Increment operator. Adds 1 to the operand; `i` is equivalent to i = i + 1.

  • += operator: Compound assignment operator that adds the right-hand operand to the left and stores the result in the left-hand operand; a += b is equivalent to a = a + b.

  • . operator: Dot operator. Left-hand operand must be an object of class type and the right-hand operand must be the name of a member of that object. The operator yields the named member of the given object.

  • :: operator: Scope operator. Among other uses, the scope operator is used to access names in a namespace. For example, std::cout denotes the name cout from the namespace std.

  • = operator: Assigns the value of the right-hand operand to the object denoted by the left-hand operand.

  • -- operator: Decrement operator. Subtracts 1 from the operand; --i is equivalent to i = i - 1.

  • << operator: Output operator. Writes the right-hand operand to the output stream indicated by the left-hand operand: cout << "hi" writes hi to the standard output. Output operations can be chained together: cout << "hi" << bye" writes hibye.

  • >> operator: Input operator. Reads from the input stream specified by the left- hand operand into the right-hand operand: cin >> i reads the next value on the standard input into i. Input operations can be chained together: cin >> i >> j reads first into i and then into j.

  • # include: Directive that makes code in a header available to a program.

  • == operator: The equality operator. Tests whether the left-hand operand is equal to the right-hand operand.

  • != operator: The inequality operator. Tests whether the left-hand operand is not equal to the right-hand operand.

  • ⇐ operator: The less-than-or-equal operator. Tests whether the left-hand operand is less than or equal to the right-hand operand.

  • < operator: The less-than operator. Tests whether the left-hand operand is less than the right-hand operand.

  • >= operator: Greater-than-or-equal operator. Tests whether the left-hand operand is greater than or equal to the right-hand operand.

  • > operator: Greater-than operator. Tests whether the left-hand operand is greater than the right-hand operand.

  • address: Number by which a byte in memory can be found.

  • alias declaration: Defines a synonym for another type: using name = type declares name as a synonym for the type type.

  • arithmetic types: Built-in types representing boolean values, characters, integers, and floating-point numbers.

  • array: Data structure that holds a collection of unnamed objects that are accessed by an index.

  • auto: Type specifier that deduces the type of a variable from its initializer.

  • base type: type specifier, possibly qualified by const, that precedes the declarators in a declaration. The base type provides the common type on which the declarators in a declaration can build.

  • bind: Associating a name with a given entity so that uses of the name are uses of the underlying entity. For example, a reference is a name that is bound to an object.

  • byte: Smallest addressable unit of memory. On most machines a byte is 8 bits.

  • class member: Part of a class.

  • compound type: A type that is defined in terms of another type.

  • const: Type qualifier used to define objects that may not be changed. const objects must be initialized, because there is no way to give them a value after they are defined.

  • const pointer: Pointer that is const.

  • const reference: Colloquial synonym for reference to const.

  • constant expression: Expression that can be evaluated at compile time.

  • constexpr: Variable that represents a constant expression.

  • conversion: Process whereby a value of one type is transformed into a value of another type. The language defines conversions among the built-in types.

  • data member: Data elements that constitute an object. Every object of a given class has its own copies of the class’ data members. Data members may be initialized when declared inside the class.

  • declaration: Asserts the existence of a variable, function, or type defined elsewhere. Names may not be used until they are defined or declared.

  • declarator: The part of a declaration that includes the name being defined and an optional type modifier.

  • decltype: Type specifier that deduces the type of a variable or an expression.

  • default initialization: How objects are initialized when no explicit initializer is given. How class type objects are initialized is controlled by the class. Objects of built-in type defined at global scope are initialized to 0; those defined at local scope are uninitialized and have undefined values.

  • definition: Allocates storage for a variable of a specified type and optionally initializes the variable. Names may not be used until they are defined or declared.

  • escape sequence: Alternative mechanism for representing characters, particularly for those without printable representations. An escape sequence is a backslash followed by a character, three or fewer octal digits, or an x followed by a hexadecimal number.

  • global scope: The scope that is outside all other scopes.

  • header guard: Preprocessor variable used to prevent a header from being included more than once in a single file.

  • identifier: Sequence of characters that make up a name. Identifiers are case- sensitive.

  • in-class initializer: Initializer provided as part of the declaration of a class data member. In-class initializers must follow an = symbol or be enclosed inside curly braces.

  • in scope: Name that is visible from the current scope.

  • initialized: A variable given an initial value when it is defined. Variables usually should be initialized.

  • inner scope: Scope that is nested inside another scope.

  • integral types: See arithmetic type.

  • list initialization: Form of initialization that uses curly braces to enclose one or more initializers.

  • literal: A value such as a number, a character, or a string of characters. The value cannot be changed. Literal characters are enclosed in single quotes, literal strings in double quotes.

  • local scope: Colloquial synonym for block scope.

  • low-level const: A const that is not top-level. Such `const`s are integral to the type and are never ignored.

  • member: Part of a class.

  • nonprintable character: A character with no visible representation, such as a control character, a backspace, newline, and so on.

  • null pointer: Pointer whose value is 0. A null pointer is valid but does not point to any object.

  • nullptr: Literal constant that denotes the null pointer.

  • object: A region of memory that has a type. A variable is an object that has a name.

  • outer scope: Scope that encloses another scope.

  • pointer: An object that can hold the address of an object, the address one past the end of an object, or zero.

  • pointer to const: Pointer that can hold the address of a const object. A pointer to const may not be used to change the value of the object to which it points.

  • preprocessor: Program that runs as part of compilation of a C++ program.

  • preprocessor variable: Variable managed by the preprocessor. The preprocessor replaces each preprocessor variable by its value before our program is compiled.

  • reference: An alias for another object.

  • reference to const: A reference that may not change the value of the object to which it refers. A reference to const may be bound to a const object, a non const object, or the result of an expression.

  • scope: The portion of a program in which names have meaning. Scopes nest. Once a name is declared, it is accessible until the end of the scope in which it was declared. C++ has several levels of scope: + global—names defined outside any other scope + class—names defined inside a class + namespace—names defined inside a namespace + block—names defined inside a block

  • separate compilation: Ability to split a program into multiple separate source files.

  • signed: Integer type that holds negative or positive values, including zero.

  • string: Library type representing variable-length sequences of characters.

  • struct: Keyword used to define a class.

  • temporary: Unnamed object created by the compiler while evaluating an expression. A temporary exists until the end of the largest expression that encloses the expression for which it was created.

  • top-level const: The const that specifies that an object may not be changed.

  • type alias: A name that is a synonym for another type. Defined through either a typedef or an alias declaration.

  • type checking: Term used to describe the process by which the compiler verifies that the way objects of a given type are used is consistent with the definition of that type.

  • type specifier: The name of a type.

  • typedef: Defines an alias for another type. When typedef appears in the base type of a declaration, the names defined in the declaration are type names.

  • undefined: Usage for which the language does not specify a meaning. Knowingly or unknowingly relying on undefined behavior is a great source of hard-to-track runtime errors, security problems, and portability problems.

  • uninitialized: Variable defined without an initial value. In general, trying to access the value of an uninitialized variable results in undefined behavior.

  • unsigned: Integer type that holds only values greater than or equal to zero.

  • variable: A named object or reference. In C++, variables must be declared before they are used.

  • void\*: Pointer type that can point to any non const type. Such pointers may not be dereferenced.

  • void type: Special-purpose type that has no operations and no value. It is not possible to define a variable of type void.

  • word: The natural unit of integer computation on a given machine. Usually a word is large enough to hold an address. On a 32-bit machine a word is typically 4 bytes.

  • & operator: Address-of operator. Yields the address of the object to which it is applied.

  • \* operator: Dereference operator. Dereferencing a pointer returns the object to which the pointer points. Assigning to the result of a dereference assigns a new value to the underlying object.

  • # define: Preprocessor directive that defines a preprocessor variable.

  • # endif: Preprocessor directive that ends an #ifdef or #ifndef region.

  • # ifdef: Preprocessor directive that determines whether a given variable is defined.

  • # ifndef: Preprocessor directive that determines whether a given variable is not defined.

C# reference and notes

Basics

// Use using to declare namespaces and functions we wish to use
// The namespaces below are all part of the standard .NET Framework Class Library
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using AnimalNS;

// But this one is not:
using System.Data.Entity;
// In order to be able to use it, you need to add a dll reference
// This can be done with the NuGet package manager: `Install-Package EntityFramework`

// Namespaces define scope to organize code into "packages" or "modules"
// Using this code from another source file: using Learning.CSharp;
namespace Learning.CSharp
{
    // Each .cs file should at least contain a class with the same name as the file.
    // You're allowed to do otherwise, but shouldn't for sanity.
    public class LearnCSharp
    {

/*
Multiline Comment
*/

/// <summary>
/// This is an XML documentation comment which can be used to generate external
/// documentation or provide context help within an IDE
/// </summary>
/// <param name="firstParam">This is some parameter documentation for firstParam</param>
/// <returns>Information on the returned value of a function</returns>
//public void MethodOrClassOrOtherWithParsableHelp(string firstParam) {}

// Give our code a custom namespace
namespace ConsoleApplication1
{
    class Program
    {
        // Code in the main function is executed
        static void Main(string[] args)
        {
            // Prints string out to the console with a line break (Write = No Line Break)
            Console.WriteLine("What is your name : ");

            // Accept input from the user
            string name = Console.ReadLine();
Types and variables
// Sbyte - Signed 8-bit integer
// (-128 <= sbyte <= 127)
sbyte fooSbyte = 100;

// Byte - Unsigned 8-bit integer
// (0 <= byte <= 255)
byte fooByte = 100;

// Short - 16-bit integer
// Signed - (-32,768 <= short <= 32,767)
// Unsigned - (0 <= ushort <= 65,535)
short fooShort = 10000;
ushort fooUshort = 10000;

// Integer - 32-bit integer
int fooInt = 1; // (-2,147,483,648 <= int <= 2,147,483,647)
uint fooUint = 1; // (0 <= uint <= 4,294,967,295)

// Long - 64-bit integer
long fooLong = 100000L; // (-9,223,372,036,854,775,808 <= long <= 9,223,372,036,854,775,807)
ulong fooUlong = 100000L; // (0 <= ulong <= 18,446,744,073,709,551,615)
// Numbers default to being int or uint depending on size.
// L is used to denote that this variable value is of type long or ulong

// Double - Double-precision 64-bit IEEE 754 Floating Point
double fooDouble = 123.4; // Precision: 15-16 digits

// Float - Single-precision 32-bit IEEE 754 Floating Point
float fooFloat = 234.5f; // Precision: 7 digits
// f is used to denote that this variable value is of type float

// Decimal - a 128-bits data type, with more precision than other floating-point types,
// suited for financial and monetary calculations
decimal fooDecimal = 150.3m;

// Boolean - true & false
bool fooBoolean = true; // or false

// Char - A single 16-bit Unicode character
char fooChar = 'A';

// Dates & Formatting
DateTime fooDate = DateTime.Now;
Console.WriteLine(fooDate.ToString("hh:mm, dd MMM yyyy"));

// Use const or read-only to make a variable immutable
// const values are calculated at compile time
const int HoursWorkPerWeek = 9001;

// The dynamic data type is defined at run time
dynamic otherName = "Paul";
otherName = 1;

// The var data type is defined when compiled and then can't change
var anotherName = "Tom";
// ERROR : anotherName = 2;
Console.WriteLine("Hello " + anotherName);

// How to get the type
Console.WriteLine("anotherName is a {0}", anotherName.GetTypeCode());
Strings
// Strings -- unlike the previous base types which are all value types,
// a string is a reference type. That is, you can set it to null
string fooString = "\"escape\" quotes and add \n (new lines) and \t (tabs)";
 Console.WriteLine(fooString);

// You can access each character of the string with an indexer:
char charFromString = fooString[1]; // => 'e'
// Strings are immutable: you can't do fooString[1] = 'X';

// Compare strings with current culture, ignoring case
string.Compare(fooString, "x", StringComparison.CurrentCultureIgnoreCase);

// Formatting, based on sprintf
string fooFs = string.Format("Check Check, {0} {1}, {0} {1:0.0}", 1, 2);

// Verbatim String
// You can use the @ symbol before a string literal to escape all characters in the string
string path = "C:\\Users\\User\\Desktop";
string verbatimPath = @"C:\Users\User\Desktop";
Console.WriteLine(path == verbatimPath);  // => true

// You can split a string over two lines with the @ symbol. To escape " use ""
string bazString = @"Here's some stuff
on a new line! ""Wow!"", the masses cried";

// Escape Sequences : \' \" \\ \b \n \t

string sampString = "A bunch of random words";

// Check if empty
Console.WriteLine("Is empty " + String.IsNullOrEmpty(sampString));
Console.WriteLine("Is empty " + String.IsNullOrWhiteSpace(sampString));
Console.WriteLine("String Length " + sampString.Length);

// Find a string index (Starts with 0)
Console.WriteLine("Index of bunch " + sampString.IndexOf("bunch"));

// Get a substring
Console.WriteLine("2nd Word " + sampString.Substring(2, 6));

string sampString2 = "More random words";

// Are strings equal
Console.WriteLine("Strings equal " + sampString.Equals(sampString2));

// Compare strings
Console.WriteLine("Starts with A bunch " + sampString.StartsWith("A bunch"));
Console.WriteLine("Ends with words " + sampString.EndsWith("words"));

// Trim white space at beginning and end or (TrimEnd / TrimStart)
sampString = sampString.Trim();

// Replace words or characters
sampString = sampString.Replace("words", "characters");
Console.WriteLine(sampString);

// Remove starting at a defined index up to the second index
sampString = sampString.Remove(0,2);
            Console.WriteLine(sampString);

// Join values in array and save to string
string[] names = new string[3] { "Matt", "Joe", "Paul" };
Console.WriteLine("Name List " + String.Join(", ", names));

// Formatting : Currency, Decimal Places, Before Decimals, Thousands Separator
string fmtStr = String.Format("{0:c} {1:00.00} {2:#.00} {3:0,0}", 1.56, 15.567, .56, 1000);

Console.WriteLine(fmtStr.ToString());

// ---------- STRINGBUILDER ----------
// Each time you create a string you actually create another string in memory
// StringBuilders are used when you want to be able to edit a string without creating new ones

StringBuilder sb = new StringBuilder();

// Append a string to the StringBuilder (AppendLine also adds a newline at the end)
sb.Append("This is the first sentence.");

// Append a formatted string
sb.AppendFormat("My name is {0} and I live in {1}", "Derek", "Pennsylvania");

// Clear the StringBuilder
// sb.Clear();

// Replaces every instance of the first with the second
sb.Replace("a", "e");

// Remove characters starting at the index and then up to the defined index
sb.Remove(5, 7);

// Out put everything
Console.WriteLine(sb.ToString());
Enums
// Enums are unique types with symbolic names and associated values
public enum Temperature
{
    Freeze,
    Low,
    Warm,
    Boil
}

Temperature micTemp = Temperature.Low;
Console.Write("What Temp : ");
Console.ReadLine();

switch (micTemp)
{
    case Temperature.Freeze:
        Console.WriteLine("Temp on Freezing");
        break;

    case Temperature.Low:
        Console.WriteLine("Temp on Low");
         break;

    case Temperature.Warm:
        Console.WriteLine("Temp on Warm");
        break;

    case Temperature.Boil:
        Console.WriteLine("Temp on Boil");
        break;
}
Structs
// A struct is a custom type that holds data made up from different data types
struct Customers
{
    private string name;
    private double balance;
    private int id;

    public void createCust(string n, double b, int i)
    {
        name = n;
        balance = b;
        id = i;
    }

    public void showCust()
    {
        Console.WriteLine("Name : " + name);
        Console.WriteLine("Balance : " + balance);
        Console.WriteLine("ID : " + id);
    }
}

Customers bob = new Customers();
bob.createCust("Bob", 15.50, 12345);
bob.showCust();
Math
Console.WriteLine("5 + 3 = " + (5 + 3));
Console.WriteLine("5 - 3 = " + (5 - 3));
Console.WriteLine("5 * 3 = " + (5 * 3));
Console.WriteLine("5 / 3 = " + (5 / 3));
Console.WriteLine("5.2 % 3 = " + (5.2 % 3));

int i = 0;

Console.WriteLine("i++ = " + (i++));
Console.WriteLine("++i = " + (++i));
Console.WriteLine("i-- = " + (i--));
Console.WriteLine("--i = " + (--i));

Console.WriteLine("i += 3 " + (i += 3));
Console.WriteLine("i -= 2 " + (i -= 2));
Console.WriteLine("i *= 2 " + (i *= 2));
Console.WriteLine("i /= 2 " + (i /= 2));
Console.WriteLine("i %= 2 " + (i %= 2));

// Math Functions
// Acos, Asin, Atan, Atan2, Cos, Cosh, Exp, Log, Sin, Sinh, Tan, Tanh
double number1 = 10.5;
double number2 = 15;

Console.WriteLine("Math.Abs(number1) " + (Math.Abs(number1)));
Console.WriteLine("Math.Ceiling(number1) " + (Math.Ceiling(number1)));
Console.WriteLine("Math.Floor(number1) " + (Math.Floor(number1)));
Console.WriteLine("Math.Max(number1, number2) " + (Math.Max(number1, number2)));
Console.WriteLine("Math.Min(number1, number2) " + (Math.Min(number1, number2)));
Console.WriteLine("Math.Pow(number1, 2) " + (Math.Pow(number1, 2)));
Console.WriteLine("Math.Round(number1) " + (Math.Round(number1)));
Console.WriteLine("Math.Sqrt(number1) " + (Math.Sqrt(number1)));

// Random Numbers
Random rand = new Random();
Console.WriteLine("Random Number Between 1 and 10 " + (rand.Next(1,11)));
Converting Data Types And Typecasting
// Convert String To Integer
// this will throw a FormatException on failure
int.Parse("123");//returns an integer version of "123"

// try parse will default to type default on failure
// in this case: 0
int tryInt;
if (int.TryParse("123", out tryInt)) // Function is boolean
                Console.WriteLine(tryInt);       // 123

// Convert Integer To String
// Convert class has a number of methods to facilitate conversions
Convert.ToString(123);
// or
tryInt.ToString();

// Casting
// Cast decimal 15 to a int
// and then implicitly cast to long
long x = (int)15M;
Operators
Console.WriteLine("\n->Operators");

int i1 = 1, i2 = 2; // Shorthand for multiple declarations

// Arithmetic is straightforward
Console.WriteLine(i1 + i2 - i1 * 3 / 7); // => 3

// Modulo
Console.WriteLine("11%3 = " + (11 % 3)); // => 2

// Comparison operators
Console.WriteLine("3 == 2? " + (3 == 2)); // => false
Console.WriteLine("3 != 2? " + (3 != 2)); // => true
Console.WriteLine("3 > 2? " + (3 > 2)); // => true
Console.WriteLine("3 < 2? " + (3 < 2)); // => false
Console.WriteLine("2 <= 2? " + (2 <= 2)); // => true
Console.WriteLine("2 >= 2? " + (2 >= 2)); // => true

// Bitwise operators!
/*
~       Unary bitwise complement
<<      Signed left shift
>>      Signed right shift
&       Bitwise AND
^       Bitwise exclusive OR
|       Bitwise inclusive OR
*/

// Incrementations
int i = 0;
Console.WriteLine("\n->Inc/Dec-rementation");
Console.WriteLine(i++); //Prints "0", i = 1. Post-Incrementation
Console.WriteLine(++i); //Prints "2", i = 2. Pre-Incrementation
Console.WriteLine(i--); //Prints "2", i = 1. Post-Decrementation
Console.WriteLine(--i); //Prints "0", i = 0. Pre-Decrementation
Control Structures
Console.WriteLine("\n->Control Structures");

// If statements are c-like
int j = 10;
if (j == 10)
{
    Console.WriteLine("I get printed");
}
else if (j > 10)
{
    Console.WriteLine("I don't");
}
else
{
    Console.WriteLine("I also don't");
}

// Ternary operators
// A simple if/else can be written as follows
// <condition> ? <true> : <false>
int toCompare = 17;
string isTrue = toCompare == 17 ? "True" : "False";

// While loop
int fooWhile = 0;
while (fooWhile < 100)
{
    //Iterated 100 times, fooWhile 0->99
    fooWhile++;
}

// Do While Loop
int fooDoWhile = 0;
do
{
    // Start iteration 100 times, fooDoWhile 0->99
    if (false)
        continue; // skip the current iteration

    fooDoWhile++;

    if (fooDoWhile == 50)
        break; // breaks from the loop completely

} while (fooDoWhile < 100);

//for loop structure => for(<start_statement>; <conditional>; <step>)
for (int fooFor = 0; fooFor < 10; fooFor++)
{
    //Iterated 10 times, fooFor 0->9
}

// For Each Loop
// foreach loop structure => foreach(<iteratorType> <iteratorName> in <enumerable>)
// The foreach loop loops over any object implementing IEnumerable or IEnumerable<T>
// All the collection types (Array, List, Dictionary...) in the .Net framework
// implement one or both of these interfaces.
// (The ToCharArray() could be removed, because a string also implements IEnumerable)
foreach (char character in "Hello World".ToCharArray())
{
    //Iterated over all the characters in the string
}

// Switch Case
// A switch works with the byte, short, char, and int data types.
// It also works with enumerated types (discussed in Enum Types),
// the String class, and a few special classes that wrap
// primitive types: Character, Byte, Short, and Integer.
int month = 3;
string monthString;
switch (month)
{
    case 1:
        monthString = "January";
        break;
    case 2:
        monthString = "February";
        break;
    case 3:
        monthString = "March";
        break;
    // You can assign more than one case to an action
    // But you can't add an action without a break before another case
    // (if you want to do this, you would have to explicitly add a goto case x
    case 6:
    case 7:
    case 8:
        monthString = "Summer time!!";
        break;
    default:
        monthString = "Some other month";
        break;
}
Arrays
// Declare an array
int[] randNumArray;

// Declare the number of items an array can contain
int[] randArray = new int[5];

// Declare and initialize an array
int[] randArray2 = { 1, 2, 3, 4, 5 };

// Arrays are mutable.
intArray[1] = 1;


// Get array length
Console.WriteLine("Array Length " + randArray2.Length);

// Get item at index
Console.WriteLine("Item 0 " + randArray2[0]);

// Cycle through array
for (int i = 0; i < randArray2.Length; i++)
{
    Console.WriteLine("{0} : {1}", i, randArray2[i]);
}

// Cycle with foreach
foreach (int num in randArray2)
{
    Console.WriteLine(num);
}

// Get the index of an item or -1
Console.WriteLine("Where is 1 " + Array.IndexOf(randArray2, 1));

string[] names = { "Tom", "Paul", "Sally" };

// Join an array into a string
string nameStr = string.Join(", ", names);
Console.WriteLine(nameStr);

// Split a string into an array
string[] nameArray = nameStr.Split(',');

// Create a multidimensional array
int[,] multArray = new int[5, 3];

// Create and initialize a multidimensional array
int[,] multArray2 = { { 0, 1 }, { 2, 3 }, { 4, 5 } };

// Cycle through multidimensional array
foreach(int num in multArray2)
{
    Console.WriteLine(num);
}

// Cycle and have access to indexes
for (int x = 0; x < multArray2.GetLength(0); x += 1)
{
    for (int y = 0; y < multArray2.GetLength(1); y += 1)
    {
        Console.WriteLine("{0} | {1} : {2}", x, y, multArray2[x, y]);
    }
}
List
// Lists are used more frequently than arrays as they are more flexible
// The format for declaring a list is follows:
// List<datatype> <var name> = new List<datatype>();

// Create a list and add values
List<int> numList = new List<int>();
numList.Add(5);
numList.Add(15);
numList.Add(25);

// Add an array to a list
int[] randArray = { 1, 2, 3, 4, 5 };
numList.AddRange(randArray);

// Clear a list
numList.Clear();

// Copy an array into a List
List<int> numList2 = new List<int>(randArray);

// Create a List with array
List<int> numList3 = new List<int>(new int[] { 1, 2, 3, 4 });

// Insert in a specific index
numList.Insert(1, 10);

// Remove a specific value
numList.Remove(5);

// Remove at an index
numList.RemoveAt(2);

// Cycle through a List with foreach or
for (int i = 0; i < numList.Count; i++)
{
    Console.WriteLine(numList[i]);
}

// Return the index for a value or -1
Console.WriteLine("4 is in index " + numList3.IndexOf(4));

// Does the List contain a value
Console.WriteLine("5 in list " + numList3.Contains(5));

// Search for a value in a string List
List<string> strList = new List<string>(new string[] { "Tom","Paul" });
Console.WriteLine("Tom in list " + strList.Contains("tom", StringComparer.OrdinalIgnoreCase));

// Sort the List
strList.Sort();

Advanced

Exception handling
// All the exceptions
// msdn.microsoft.com/en-us/library/system.systemexception.aspx#inheritanceContinued

try
{
    Console.Write("Divide 10 by ");
    int num = int.Parse(Console.ReadLine());
    Console.WriteLine("10 / {0} =  {1}", num, (10/num));
}

// Specifically catches the divide by zero exception
catch (DivideByZeroException ex)
{
    Console.WriteLine("Can't divide by zero");

    // Get additonal info on the exception
    Console.WriteLine(ex.GetType().Name);
    Console.WriteLine(ex.Message);

    // Throw the exception to the next inline
    // throw ex;

    // Throw a specific exception
    throw new InvalidOperationException("Operation Failed", ex);
}

// Catches any other exception
catch (Exception ex)
{
    Console.WriteLine("An error occurred");
    Console.WriteLine(ex.GetType().Name);
    Console.WriteLine(ex.Message);
}
Anonymous methods and lambda expressions
            // An anonymous method has no name and its return type is defined by the return used in the method

            GetSum sum = delegate (double num1, double num2) {
                return num1 + num2;
            };

            Console.WriteLine("5 + 10 = " + sum(5, 10));

            // ---------- LAMBDA EXPRESSIONS ----------
            // A lambda expression is used to act as an anonymous function or expression tree

            // You can assign the lambda expression to a function instance
            Func<int, int, int> getSum = (x, y) => x + y;
            Console.WriteLine("5 + 3 = " + getSum.Invoke(5, 3));

            // Get odd numbers from a list
            List<int> numList = new List<int> { 5, 10, 15, 20, 25 };

            // With an Expression Lambda the input goes in the left (n) and the statements go on the right
            List<int> oddNums = numList.Where(n => n % 2 == 1).ToList();

            foreach (int num in oddNums) {
                Console.Write(num + ", ");
            }
File I/O
// The StreamReader and StreamWriter allows you to create text files while reading and
// writing to them

string[] custs = new string[] { "Tom", "Paul", "Greg" };

using (StreamWriter sw = new StreamWriter("custs.txt"))
{
    foreach(string cust in custs)
    {
        sw.WriteLine(cust);
    }
}

string custName = "";
using (StreamReader sr = new StreamReader("custs.txt"))
    {
        while ((custName = sr.ReadLine()) != null)
        {
                Console.WriteLine(custName);
         }
    }

Console.Write("Hit Enter to Exit");
string exitApp = Console.ReadLine();
classes
Animal bulldog = new Animal(13, 50, "Spot", "Woof");
Console.WriteLine("{0} says {1}", bulldog.name, bulldog.sound);
Console.WriteLine("No. of Animals " + Animal.getNumOfAnimals());

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ConsoleApplication2
{
    class Animal
    {

        // public : Access is not limited
        // protected : Access is limited to the class methods and subclasses
        // private : Access is limited to only this classes methods
        public double height { get; set; }
        public double weight { get; set; }
        public string sound { get; set; }

        // We can either have C# create the getters and setters or create them ourselves to verify data
        public string name;
        public string Name
        {
            get { return name; }
            set { name = value; }
        }

        // Every object has a default constructor that receives no attributes
        // The constructor initializes every object created
        // this is used to refer to this objects specific fields since we don't know the objects given name

        // The default constructor isn't created if you create any other constructor
        public Animal()
        {
            this.height = 0;
            this.weight = 0;
            this.name = "No Name";
            this.sound = "No Sound";

            numOfAnimals++;
        }

        // You can create custom constructors as well
        public Animal(double height, double weight, string name, string sound)
        {
            this.height = height;
            this.weight = weight;
            this.name = name;
            this.sound = sound;

            numOfAnimals++;
        }

        // A static fields value is shared by every object of the Animal class
        // We declare thinsg static when it doesn't make sense for our object to either have the property or
        // the capability to do something (Animals can't count)
        static int numOfAnimals = 0;

        // A static method cannot access non-static class members
        public static int getNumOfAnimals()
        {
            return numOfAnimals;
        }

        // Declare a method
        public string toString()
        {
            return String.Format("{0} is {1} inches tall, weighs {2} lbs and likes to say {3}", name, height, weight, sound);
        }

        // Overloading methods works if you have methods with different attribute data types
        // You can give attributes default values
        public int getSum(int num1 = 1, int num2 = 1)
        {
            return num1 + num2;
        }

        public double getSum(double num1, double num2)
        {
            return num1 + num2;
        }

        static void Main(string[] args)
        {
            // Create an Animal object and call the constructor
            Animal spot = new Animal(15, 10, "Spot", "Woof");

            // Get object values with the dot operator
            Console.WriteLine("{0} says {1}", spot.name, spot.sound);

            // Calling a static method
            Console.WriteLine("Number of Animals " + Animal.getNumOfAnimals());

            // Calling an object method
            Console.WriteLine(spot.toString());

            Console.WriteLine("3 + 4 = " + spot.getSum(3, 4));

            // You can assign attributes by name
            Console.WriteLine("3.4 + 4.5 = " + spot.getSum(num2: 3.4, num1: 4.5));

            // You can create objects with an object initializer
            Animal grover = new Animal
            {
                name = "Grover",
                height = 16,
                weight = 18,
                sound = "Grrr"
            };

            Console.WriteLine(grover.toString());

            // Create a subclass Dog object
            Dog spike = new Dog();

            Console.WriteLine(spike.toString());

            spike = new Dog(20, 15, "Spike", "Grrr Woof", "Chicken");

            Console.WriteLine(spike.toString());

            // One way to implement polymorphism is through an abstract class
            Shape rect = new Rectangle(5, 5);
            Shape tri = new Triangle(5, 5);
            Console.WriteLine("Rect Area " + rect.area());
            Console.WriteLine("Trit Area " + tri.area());

            // Using the overloaded + on 2 Rectangles
            Rectangle combRect = new Rectangle(5, 5) + new Rectangle(5, 5);

            Console.WriteLine("combRect Area = " + combRect.area());

            // ---------- GENERICS ----------
            // With Generics you don't have to specify the data type of an element in a class or method
            KeyValue<string, string> superman = new KeyValue<string, string>("","");
            superman.key = "Superman";
            superman.value = "Clark Kent";
            superman.showData();

            // Now use completely different types
            KeyValue<int, string> samsungTV = new KeyValue<int, string>(0, "");
            samsungTV.key = 123456;
            samsungTV.value = "a 50in Samsung TV";
            samsungTV.showData();

            Console.Write("Hit Enter to Exit");
            string exitApp = Console.ReadLine();

        }
    }

    class Dog : Animal
    {
        public string favFood { get; set; }

        // Set the favFood default and then call the Animal super class constructor
        public Dog() : base()
        {
            this.favFood = "No Favorite Food";
        }

        public Dog(double height, double weight, string name, string sound, string favFood) :
            base(height, weight, name, sound)
        {
            this.favFood = favFood;
        }

        // Override methods with the keyword new
        new public string toString()
        {
            return String.Format("{0} is {1} inches tall, weighs {2} lbs, likes to say {3} and eats {4}", name, height, weight, sound, favFood);
        }

    }

    // Abstract classes define methods that must be defined by derived classes
    // You can only inherit one abstract class per class
    // You can't instantiate an abstract class
    abstract class Shape
    {
        public abstract double area();

        // An abstract class can contain complete or default code for methods
        public void sayHi()
        {
            Console.WriteLine("Hello");
        }
    }

    // A class can have many interfaces
    // An interface can't have concrete code
    public interface ShapeItem
    {
        double area();
    }

    class Rectangle : Shape
    {
        private double length;
        private double width;

        public Rectangle( double num1, double num2)
        {
            length = num1;
            width = num2;
        }

        public override double area()
        {
            return length * width;
        }

        // You can redefine many built in operators so that you can define what happens when you
        // add to Rectangles
        public static Rectangle operator+ (Rectangle rect1, Rectangle rect2)
        {
            double rectLength = rect1.length + rect2.length;
            double rectWidth = rect1.width + rect2.width;

            return new Rectangle(rectLength, rectWidth);

        }

    }

    class Triangle : Shape
    {
        private double theBase;
        private double height;

        public Triangle(double num1, double num2)
        {
            theBase = num1;
            height = num2;
        }

        public override double area()
        {
            return .5 * (theBase * height);
        }
    }

    // ---------- GENERIC CLASS ----------

    class KeyValue<TKey, TValue>
    {
        public TKey key { get; set; }
        public TValue value { get; set; }

        public KeyValue(TKey k, TValue v)
        {
            key = k;
            value = v;
        }

        public void showData()
        {
            Console.WriteLine("{0} is {1}", this.key, this.value);
        }

    }

}

Python reference and notes

Types and Operations

Comments
# Single line comments start with a number symbol.

""" Multiline strings can be written
    using three "s, and are often used
    as documentation.
"""
Main data types
boolean = True / False
integer = 10
float = 10.01
string = "123abc"
list = [ value1, value2, … ]
dictionary = { key1:value1, key2:value2, …}
Operators
+ addition
- subtraction
* multiplication
/ division
** exponent
% modulus
// floor division

# Result of integer division truncated down both for positive and negative.
5 // 3       # => 1
5.0 // 3.0   # => 1.0 # works on floats too
-5 // 3      # => -2
-5.0 // 3.0  # => -2.0

# The result of division is always a float
10.0 / 3  # => 3.3333333333333335

(9 / 3), (9.0 / 3), (9 // 3), (9 // 3.0) # => (3.0, 3.0, 3, 3.0)

Bitwise Operations

# As a rule of thumb, if you find yourself wanting to flip bits in
# Python, you should think about which language you’re really coding.

x = 1  # 1 decimal is 0001 in bits
x << 2 # Shift left 2 bits: 0100 => 4
x | 2  # Bitwise OR (either bit=1): 0011 => 3
x & 1  # Bitwise AND (both bits=1): 0001 => 1

# bit strings and hex literals, Learning python, 5th ed, page 154
Comparison & booleans:
== equal
!= different
> higher
< lower
>= higher or equal
<= lower or equal

# Comparisons can be chained!
1 < 2 < 3  # => True
2 < 3 < 2  # => False

and logical AND
or logical OR
not logical NOT

# (is vs. ==) is checks if two variables refer to the same object, but == checks
# if the objects pointed to have the same values.
a = [1, 2, 3, 4]  # Point a at a new list, [1, 2, 3, 4]
b = a             # Point b at what a is pointing to
b is a            # => True, a and b refer to the same object
b == a            # => True, a's and b's objects are equal
b = [1, 2, 3, 4]  # Point b at a new list, [1, 2, 3, 4]
b is a            # => False, a and b do not refer to the same object
b == a            # => True, a's and b's objects are equal

# None is an object
None  # => None

# Don't use the equality "==" symbol to compare objects to None
# Use "is" instead. This checks for equality of object identity.
"etc" is None  # => False
None is None   # => True

# None, 0, and empty strings/lists/dicts/tuples all evaluate to False.
# All other values are True
bool(0)   # => False
bool("")  # => False
bool([])  # => False
bool({})  # => False
bool(())  # => False
Variables
# There are no declarations, only assignments.
# Convention is to use lower_case_with_underscores
some_var = 5
some_var  # => 5

# Accessing a previously unassigned variable is an exception.
# See Control Flow to learn more about exception handling.
some_unknown_var  # Raises a NameError

# if can be used as an expression
# Equivalent of C's '?:' ternary operator
"yahoo!" if 3 > 2 else 2  # => "yahoo!"
Special characters
#   comment
\   multiline commands joining
\n  Newline
\\	Backslash (\)
\'	Single quote (')
\"	Double quote (")
\a	ASCII Bell (BEL)
\b	ASCII Backspace (BS)
\f	ASCII Formfeed (FF)
\n	ASCII Linefeed (LF)
\r	ASCII Carriage Return (CR)
\t	ASCII Horizontal Tab (TAB)
\v	ASCII Vertical Tab (VT)
\ooo	Character with octal value ooo	(1,3)
\xhh	Character with hex value hh
Console input/output
print([object, ...][, sep=' '][, end='\n'][, file=sys.stdout][, flush=False])

print("hi") # => hi
print("there are", 365, "days in year.") # => there are 365 days in year.

# By default the print function also prints out a newline at the end.
# Use the optional argument end to change the end string.
print("Hello, World", end="!")  # => Hello, World!
print("hmm", end=' ') # output space instead of newline

lst = [1,2,3]
print(*lst, sep='|') # print list elements seperated by | # => 1|2|3

# Simple way to get input data from console, returns data as string
# Note: In earlier versions of Python, input() method was named as raw_input()
name = input("enter your name: ")
num = int(input("enter a number: "))

print('-' * 80) # 80 dashes

#print to file
print('Hello World', file=open('file.txt', 'w'))
Strings
'hm' # single quoted
"hmmm" # double quoted
'''multi
line''' # multi line string
r'\temp\spam' # raw string (no escapes)
b'sp\xc4m' # byte string
u'sp\u00c4m' # Unicode string

len('hello') # string length => 5

msg = 'hello'
if msg < 'n': print('a-m') # strings are compared character at a time...
else: print ('n-z') # ...in lexicographic order. => a-m

'e' in msg # returns boolean => True


##### Sequence Operations ####
'hello' + 'world' # concentrate strings => helloworld
'Tom'*2 # multiply string by integer => TomTom

msg[-1] # The last item in msg => 'o'
msg[len(S)-1] # Negative indexing, the hard way => 'o'
msg[:] # All of S as a top-level copy (0:len(S)) => hello

# A string can be treated like a list of characters
"test"[2] # => s
string = "This is a string"
string[0] # => 'T'

string[0:4] # => 'This'

temp = ['s','p','a','m']
",".join(temp) # => 's,p,a,m'

spam = "spam"
spam[1:3], spam[1:], spam[:-1] # => ('pa', 'pam', 'spa')
spam = spam + 'Burger' # To change a string, make a new one => SpamBurger

S = 'abcdefghijklmnop'
S[1:10:2] # => 'bdfhj'
S[::2] # => 'acegikmo'

# reverse a sequence
"hello"[::−1] # => 'olleh'

##### Immutability ####
# Strings are immutable.
string[0] = 'z'  # Immutable objects cannot be changed
...error text omitted...
TypeError: 'str' object does not support item assignment
string = 'z' + S[1:]
string # => 'zpam'

## Changing text in place using list ##
S = 'shrubbery'
L = list(S) # Expand to a list: [...]
L # => ['s', 'h', 'r', 'u', 'b', 'b', 'e', 'r', 'y']
L[1] = 'c' # Change it in place
''.join(L) # Join with empty delimiter => 'scrubbery'

## Changing text in place using bytearray ##
B = bytearray(b'spam') # A bytes/list hybrid (ahead)
B.extend(b'eggs') # 'b' needed in 3.X, not 2.X
B # B[i] = ord(c) works here too
bytearray(b'spameggs')
B.decode() # Translate to normal string => 'spameggs'


##### Type-Specific Methods ####
string.find('is') # Find the offset of a substring => 2
string.replace(" ", "-") # => 'This-is-a-string'
string # => 'This is a string'

line = 'aaa,bbb,ccccc,dd'
# Split on a delimiter into a list of substrings
line.split(',') # => ['aaa', 'bbb', 'ccccc', 'dd']

S = 'spam'
# Upper- and lowercase conversions
S.upper() # => 'SPAM'
# Content tests: isalpha, isdigit, etc.
S.isalpha() # => True

line = 'aaa,bbb,ccccc,dd\n'
# Remove whitespace characters on the right side
line.rstrip() # => 'aaa,bbb,ccccc,dd'
# Combine two operations
line.rstrip().split(',') # => ['aaa', 'bbb', 'ccccc', 'dd']

## Formating ##
"hello {}".format('world') # => hello world

# You can repeat the formatting arguments to save some typing.
"{0} be nimble, {0} be quick, {0} jump over the {1}".format("Jack", "candle stick")
# => "Jack be nimble, Jack be quick, Jack jump over the candle stick"

# You can use keywords if you don't want to count.
"{name} wants to eat {food}".format(name="Bob", food="lasagna")
# => "Bob wants to eat lasagna"

# Separators, decimal digits
'{:,.2f}'.format(296999.2567) # => '296,999.26'

# Digits, padding, signs
'%.2f | %+05d' % (3.14159, −42) # => '3.14 | −0042'

# Dictionary based formating
reply = """
Greetings...
Hello %(name)s!
Your age is %(age)s
"""
values = {'name': 'Bob', 'age': 40}
print(reply % values)
# Greetings...
# Hello Bob!
# Your age is 40

import sys
'My {1[kind]} runs {0.platform}'.format(sys, {'kind': 'pc'}) # => 'My pc runs linux'
'{.platform:>10} = {[kind]:<10}'.format(sys, dict(kind='laptop')) # => '    win32 = laptop    '

data = dict(platform=sys.platform, kind='laptop')
'My {kind:<8} runs {platform:>8}'.format(**data) # => 'My laptop   runs    linux'

'{0:,d}'.format(999999999999) # => '999,999,999,999'
'{:,.2f}'.format(296999.2567) # => '296,999.26'

from formats import commas, money
'%s' % money(296999.2567) # => '$296,999.26'
[commas(x) for x in (9999999, 8888888)] # => ['9,999,999', '8,888,888']


##### Getting help ####
dir(string) # => ['__add__', '__class__', ... 'upper', 'zfill']
help(sting.replace) #=> Help on built-in function replace:\n replace(...)\n ...
Lists
##### Sequence Operations ####
li = []  # Lists store sequences
other_li = [4, 5, 6] # You can start with a prefilled list

# Access a list like you would any array
li[0]
# Look at the last element
li[-1]
# Looking out of bounds is an IndexError
li[4]  # Raises an IndexError

# You can look at ranges with slice syntax.
# The start index is included, the end index is not
# (It's a closed/open range for you mathy types.)
li = [2,3,4,5]
li[1:3]  # => [3, 4]
li[2:]   # Omit the beginning and return the list
li[:3]   # Omit the end and return the list
li[:-1]  # return all but the last element
li[::2]  # Select every second entry
li[::-1] # Return a reversed copy of the list
# Use any combination of these to make advanced slices li[start:end:step]

# Make a one layer deep copy using slices
li2 = li[:]  # (li2 is li) will result in false.

li[i:j] = otherlist  # replace ith to jth element with otherlist

L = ['spam', 'Spam', 'SPAM!']
L[0:2] = ['eat', 'more']
L # => ['eat', 'more', 'SPAM!']


##### Nesting ####
M = [[1, 2, 3], # A 3 × 3 matrix, as nested lists
     [4, 5, 6], # Code can span lines if bracketed
     [7, 8, 9]]
M # => [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

M[1] # Get row 2
# => [4, 5, 6]
M[1][2] # Get row 2, then get item 3 within the row
# => 6


##### Type Specific Operations ####
# Add stuff to the end of a list with append
li = []
li.append(1)    # li is now [1]
li.append(2)    # li is now [1, 2]
li.append(3)    # li is now [1, 2, 3]

# Remove from the end with pop
li.pop()        # => 3 and li is now [1, 2]

del li[1]   # remove 1
del li[i:j] # delete range

# Remove first occurrence of a value
li.remove(2)  # li is now [1, 3]
li.remove(2)  # Raises a ValueError as 2 is not in the list

# Insert an element at a specific index
li.insert(1, 2)  # li is now [1, 2, 3] again

# Get the index of the first item found matching the argument
li.index(2)  # => 1
li.index(4)  # Raises a ValueError as 4 is not in the list

# You can add lists
# Note: values for li and for other_li are not modified.
li + other_li  # => [1, 2, 3, 4, 5, 6]

# Concatenate lists with "extend()"
li.extend(other_li)  # Now li is [1, 2, 3, 4, 5, 6]

len(li)      # Examine the length with "len()"
set(li)      # remove duplicate elements from a list
li.sort()    # sort list items
li.clear()   # removes all items from the list
li.reverse()
li.count(item)
sum(li)

zip(list1, list2) #  zip is a container, returns iterator
sorted(li)        # returns sorted list but doesn't change list itself
",".join(li)      # join list elements with ,

# Check for existence in a list with "in"
1 in li  # => True
Dictionaries
# Dictionaries store mappings from keys to values
empty_dict = {}
# Here is a prefilled dictionary
filled_dict = {"one": 1, "two": 2, "three": 3}

bob1 = dict(name='Bob', job='dev', age=40) # Keywords
bob1 # => {'age': 40, 'name': 'Bob', 'job': 'dev'}

bob2 = dict(zip(['name', 'job', 'age'], ['Bob', 'dev', 40])) # Zipping
bob2 # => {'job': 'dev', 'name': 'Bob', 'age': 40}

# Adding to a dictionary
filled_dict.update({"four":4})  # => {"one": 1, "two": 2, "three": 3, "four": 4}
filled_dict["four"] = 4         # another way to add to dict
filled_dict # => {'two': 2, 'three': 3, 'four': 4, 'one': 1}

# From Python 3.5 you can also use the additional unpacking options
{'a': 1, **{'b': 2}}  # => {'a': 1, 'b': 2}
{'a': 1, **{'a': 2}}  # => {'a': 2}

# Note keys for dictionaries have to be immutable types. This is to ensure that
# the key can be converted to a constant hash value for quick look-ups.
# Immutable types include ints, floats, strings, tuples.
invalid_dict = {[1,2,3]: "123"}  # => Raises a TypeError: unhashable type: 'list'
valid_dict = {(1,2,3):[1,2,3]}   # Values can be of any type, however.

# Look up values with []
# works for keys, throws KeyError for values
filled_dict["one"]  # => 1
# Use "get()" method to avoid the KeyError
filled_dict.get("one")      # => 1
filled_dict.get("four")     # => None
# The get method supports a default argument when the value is missing
filled_dict.get("one", 4)   # => 1
filled_dict.get("four", 4)  # => 4

# Get all keys as an iterable with "keys()". We need to wrap the call in list()
# to turn it into a list. We'll talk about those later.  Note - Dictionary key
# ordering is not guaranteed. Your results might not match this exactly.
list(filled_dict.keys())  # => ["three", "two", "one"]

# Get all values as an iterable with "values()". Once again we need to wrap it
# in list() to get it out of the iterable. Note - Same as above regarding key ordering.
list(filled_dict.values())  # => [3, 2, 1]

# Check for existence of keys in a dictionary with "in"
"one" in filled_dict  # => True
1 in filled_dict      # => False

# "setdefault()" inserts into a dictionary only if the given key isn't present
filled_dict.setdefault("five", 5)  # filled_dict["five"] is set to 5
filled_dict.setdefault("five", 6)  # filled_dict["five"] is still 5

# Remove keys from a dictionary with del
del filled_dict["one"]  # Removes the key "one" from filled dict

# pop() removes the item associated to the key and returns its value
filled_dict.pop("four") # => 4

# sorting
D = {'a': 1, 'b': 2, 'c': 3}
KS = list(D.keys()) # Unordered keys list
KS # => ['c', 'a', 'b']
KS.sort() # sorted keys list
KS # => ['a', 'b', 'c']

for key in sorted(D):
    print(key, '=>', D[key])
# a => 1
# b => 2
# c => 3


dict.items() # returns a list of pairs (key,value)
dict.clear() # removes all keys-values from the dictionary
dict.copy()  # returns a copy of the dictionary
Tuples
# Tuples are like lists but are immutable.
tup = (1, 2, 3)
tup[0]      # => 1
tup[0] = 3  # Raises a TypeError

# Note that a tuple of length one has to have a comma after the last element but
# tuples of other lengths, even zero, do not.
type((1))   # => <class 'int'>
type((1,))  # => <class 'tuple'>
type(())    # => <class 'tuple'>

# Tuples also have type-specific callable methods
tup.index(2) # => 2; 2 appears at offset 1
tup.count(1) # => 1; 1 appears only once

# You can do most of the list operations on tuples too
len(tup)         # => 3
tup + (4, 5, 6)  # => (1, 2, 3, 4, 5, 6)
tup[:2]          # => (1, 2)
2 in tup         # => True

# Like lists and dictionaries, tuples support mixed types and nesting,
# but they don’t grow and shrink because they are immutable
T = 'spam', 3.0, [11, 22, 33]
T[1] # => 3
T[2][1] # => 22
T.append(4) # => AttributeError: 'tuple' object has no attribute 'append'

# sorting
T = ('cc', 'aa', 'dd', 'bb')
tmp = list(T)
tmp.sort()
T = tuple(tmp)
R # => ('aa', 'bb', 'cc', 'dd')
sorted(T) # Or use the sorted built-in, and save two steps
# => ['aa', 'bb', 'cc', 'dd']


# You can unpack tuples (or lists) into variables
a, b, c = (1, 2, 3)  # a is now 1, b is now 2 and c is now 3
# You can also do extended unpacking
a, *b, c = (1, 2, 3, 4)  # a is now 1, b is now [2, 3] and c is now 4
# Tuples are created by default if you leave out the parentheses
d, e, f = 4, 5, 6
# Now look how easy it is to swap two values
e, d = d, e  # d is now 5 and e is now 4


# named tuples - tuple/class/dictionary hybrid
from collections import namedtuple               # Import extension type
Rec = namedtuple('Rec', ['name', 'age', 'jobs']) # Make a generated class
bob = Rec('Bob', age=40.5, jobs=['dev', 'mgr'])  # A named-tuple record
bob # => Rec(name='Bob', age=40.5, jobs=['dev', 'mgr'])
bob[0], bob.jobs # Access by position and attribute respectively
# => ('Bob', ['dev', 'mgr'])

O = bob._asdict()    # Dictionary-like form
O['name'], O['jobs'] # Access by key too => ('Bob', ['dev', 'mgr'])
0 # => OrderedDict([('name', 'Bob'), ('age', 40.5), ('jobs', ['dev', 'mgr'])])
Files
# Writhing to file
f = open('data.txt', 'w') # Make a new file in output mode ('w' is write)
f.write('Hello\n')  # => 6 Return number of items written in Python 3.X
f.write('world\n')
f.close() #lClose to flush output buffers to disk


# Reading file
f = open('data.txt') # 'r' (read) is the default processing mode
text = f.read() # Read entire file into a string
text # => 'Hello\nworld\n'

print(text) # print interprets control characters
# Hello
# world

text.split() # File content is always a string
# => ['Hello', 'world']


# File Context Managers
for line in open('data.txt'): print(line) # safer way to deal with files

with open(r'C:\code\data.txt') as myfile: # safer way to deal with files
    for line in myfile:
        ...use line here...


# CVS
import csv
rdr = csv.reader(open('csvdata.txt'))
for row in rdr: print(row)
# ...
# ['a', 'bbb', 'cc', 'dddd']
# ['11', '22', '33', '44']


# Pickle: storing native python objects
D = {'a': 1, 'b': 2}
F = open('datafile.pkl', 'wb')
import pickle
pickle.dump(D, F)  # Pickle any object to file
F.close()

F = open('datafile.pkl', 'rb')
E = pickle.load(F) # Load any object from file
E # => {'a': 1, 'b': 2}


# Storing objects in JSON
name = dict(first='Bob', last='Smith')
rec = dict(name=name, job=['dev', 'mgr'], age=40.5)
rec # => {'job': ['dev', 'mgr'], 'name': {'last': 'Smith', 'first': 'Bob'}, 'age': 40.5}

import json
json.dump(rec, fp=open('testjson.txt', 'w'), indent=4)
print(open('testjson.txt').read())
#{
#    "job": [
#        "dev", . . .
P = json.load(open('testjson.txt'))
P # => {'job': ['dev', 'mgr'], 'name': {'last': 'Smith', 'first': 'Bob'}, 'age': 40.5}


# Storing Packed Binary Data: struct
# packing
F = open('data.bin', 'wb')  # Open binary output file
import struct
data = struct.pack('>i4sh', 7, b'spam', 8) # Make packed binary data
data # => b'\x00\x00\x00\x07spam\x00\x08'
F.write(data) # Write byte string
F.close()

# unpacking
F = open('data.bin', 'rb'); data = F.read() # Get packed binary data
values = struct.unpack('>i4sh', data)
values # => (7, b'spam', 8)
Sets
empty_set = set() # Sets store...sets
# Initialize a set with a bunch of values. Yeah, it looks a bit like a dict. Sorry.
some_set = {1, 1, 2, 2, 3, 4}  # some_set is now {1, 2, 3, 4}

# Similar to keys of a dictionary, elements of a set have to be immutable.
invalid_set = {[1], 1}  # => Raises a TypeError: unhashable type: 'list'
valid_set = {(1,), 1}

# Add one more item to the set
filled_set = some_set
filled_set.add(5)  # filled_set is now {1, 2, 3, 4, 5}

# Do set intersection with &
other_set = {3, 4, 5, 6}
filled_set & other_set  # => {3, 4, 5}
{1, 2, 3}.intersection((1, 3, 5)) # => {1,3}

# Do set union with |
filled_set | other_set  # => {1, 2, 3, 4, 5, 6}
{1, 2, 3}.union([3, 4]) # => {1, 2, 3, 4}

# Do set difference with -
{1, 2, 3, 4} - {2, 3, 5}  # => {1, 4}

# Do set symmetric difference with ^
{1, 2, 3, 4} ^ {2, 3, 5}  # => {1, 4, 5}

# Check if set on the left is a superset of set on the right
{1, 2} >= {1, 2, 3} # => False

# Check if set on the left is a subset of set on the right
{1, 2} <= {1, 2, 3} # => True
{1, 2, 3}.issubset(range(-5, 5)) # => True

# Check for existence in a set with in
2 in filled_set   # => True
10 in filled_set  # => False

# useful example
engineers = {'bob', 'sue', 'ann', 'vic'}
managers = {'tom', 'sue'}

'bob' in engineers # Is bob an engineer? =>True
engineers & managers # Who is both engineer and manager? => {'sue'}
managers - engineers # Managers who are not engineers => {'tom'}

Statements and syntax

flow control
# Let's just make a variable
some_var = 5

# Here is an if statement. Indentation is significant in Python!
# Convention is to use four spaces, not tabs.
# This prints "some_var is smaller than 10"
if some_var > 10:
    print("some_var is totally bigger than 10.")
elif some_var < 10:    # This elif clause is optional.
    print("some_var is smaller than 10.")
else:                  # This is optional too.
    print("some_var is indeed 10.")
Iteration
for item in ["a", "b", "c"]:  # item becoms a, b, c
for i in range(4):            # 0 to 3
for i in range(4, 8):         # 4 to 7
for i in range(0, 8, 2):      # 0,2,4,6
for key, val in dict.items(): # gets key,val from dict
for c in "hacker": print(c, end=' ') # => h a c k e r
for c in 'spam': print(c.upper(), end="") # => SPAM

x = 0 # While loops go until a condition is no longer met.
while x < 4:
    print(x)
    x += 1  # Shorthand for x = x + 1
Iterables
# Python offers a fundamental abstraction called the Iterable.
# An iterable is an object that can be treated as a sequence.
# The object returned by the range function, is an iterable.
filled_dict = {"one": 1, "two": 2, "three": 3}
our_iterable = filled_dict.keys()
print(our_iterable)  # => dict_keys(['one', 'two', 'three']). This is an object that implements our Iterable interface.

# We can loop over it.
for i in our_iterable:
    print(i)  # Prints one, two, three

# However we cannot address elements by index.
our_iterable[1]  # Raises a TypeError

# An iterable is an object that knows how to create an iterator.
our_iterator = iter(our_iterable)

# Our iterator is an object that can remember the state as we traverse through it.
# We get the next object with "next()".
next(our_iterator)  # => "one"

# It maintains state as we iterate.
next(our_iterator)  # => "two"
next(our_iterator)  # => "three"

# After the iterator has returned all of its data, it raises a StopIteration exception
next(our_iterator)  # Raises StopIteration

# You can grab all the elements of an iterator by calling list() on it.
list(filled_dict.keys())  # => Returns ["one", "two", "three"]

Functions, generators, modules and packages

Functions
# Use "def" to create new functions
def add(x, y):
    print("x is {} and y is {}".format(x, y))
    return x + y  # Return values with a return statement

# Calling functions with parameters
add(5, 6)  # => prints out "x is 5 and y is 6" and returns 11

# Another way to call functions is with keyword arguments
add(y=6, x=5)  # Keyword arguments can arrive in any order.

# You can define functions that take a variable number of
# positional arguments
def varargs(*args):
    return args

varargs(1, 2, 3)  # => (1, 2, 3)

# You can define functions that take a variable number of
# keyword arguments, as well
def keyword_args(**kwargs):
    return kwargs

keyword_args(big="foot", loch="ness")  # => {"big": "foot", "loch": "ness"}

# You can do both at once, if you like
def all_the_args(*args, **kwargs):
    print(args)
    print(kwargs)

all_the_args(1, 2, a=3, b=4) # => (1, 2) \n {"a": 3, "b": 4}


# When calling functions, you can do the opposite of args/kwargs!
# Use * to expand tuples and use ** to expand kwargs.
args = (1, 2, 3, 4)
kwargs = {"a": 3, "b": 4}
all_the_args(*args)            # equivalent to all_the_args(1, 2, 3, 4)
all_the_args(**kwargs)         # equivalent to all_the_args(a=3, b=4)
all_the_args(*args, **kwargs)  # equivalent to all_the_args(1, 2, 3, 4, a=3, b=4)

# Returning multiple values (with tuple assignments)
def swap(x, y):
    return y, x  # Return multiple values as a tuple without the parenthesis.
                 # (Note: parenthesis have been excluded but can be included)

x = 1
y = 2
x, y = swap(x, y)     # => x = 2, y = 1
# (x, y) = swap(x,y)  # Again parenthesis have been excluded but can be included.
#you can also swap values as following
x, y = y, x

# Function Scope
x = 5
def set_x(num):
    # Local var x not the same as global variable x
    x = num    # => 43
    print(x)   # => 43

def set_global_x(num):
    global x
    print(x)   # => 5
    x = num    # global var x is now set to 6
    print(x)   # => 6

set_x(43)
set_global_x(6)


# Python has first class functions
def create_adder(x):
    def adder(y):
        return x + y
    return adder

add_10 = create_adder(10)
add_10(3)   # => 13
Anonymous functions
(lambda x: x > 2)(3)                  # => True
(lambda x, y: x ** 2 + y ** 2)(2, 1)  # => 5

# There are built-in higher order functions
list(map(add_10, [1, 2, 3]))          # => [11, 12, 13]
list(map(max, [1, 2, 3], [4, 2, 1]))  # => [4, 2, 3]

list(filter(lambda x: x > 5, [3, 4, 5, 6, 7]))  # => [6, 7]

# We can use list comprehensions for nice maps and filters
# List comprehension stores the output as a list which can itself be a nested list
[add_10(i) for i in [1, 2, 3]]         # => [11, 12, 13]
[x for x in [3, 4, 5, 6, 7] if x > 5]  # => [6, 7]

# You can construct set and dict comprehensions as well.
{x for x in 'abcddeef' if x not in 'abc'}  # => {'d', 'e', 'f'}
{x: x**2 for x in range(5)}  # => {0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
Comprehensions
# Comprehensions are constructs that allow sequences to be built from other
# sequences. Python 2.0 introduced list comprehensions and Python 3.0 comes with
# dictionary and set comprehensions.

# Say we need to obtain a list of all the integers in a sequence and then square them:
a_list = [1, '4', 9, 'a', 0, 4]
squared_ints = [ e**2 for e in a_list if type(e) == types.IntType ]
print(squared_ints) # [ 1, 81, 0, 16 ]

# Much the same results can be achieved using the built in functions,
# map, filter and the anonymous lambda function.
# Note that function calls in Python are expensive.
#The filter function applies a predicate to a sequence:
filter(lambda e: type(e) == types.IntType, a_list)
#Map modifies each member of a sequence:
map(lambda e: e**2, a_list)
# The two can be combined:
map(lambda e: e**2, filter(lambda e: type(e) == types.IntType, a_list))
Generators
Decorators
Modules
# Python modules are just ordinary Python files. You
# can write your own, and import them. The name of the
# module is the same as the name of the file.

# You can import modules
import math
print(math.sqrt(16))  # => 4.0

# You can get specific functions from a module
from math import ceil, floor
print(ceil(3.7))   # => 4.0
print(floor(3.7))  # => 3.0

# You can import all functions from a module. not recommended
from math import *

# You can shorten module names
import math as m
math.sqrt(16) == m.sqrt(16)  # => True

# You can find out which functions and attributes
# are defined in a module.
import math
dir(math)

# If you have a Python script named math.py in the same
# folder as your current script, the file math.py will
# be loaded instead of the built-in Python module.
# This happens because the local folder has priority
# over Python's built-in libraries.

# commonly used builtin modules
math.floor(d) #floor to next lower integer
math.trunc(d) #drop decimal digits

import random
random.random() #random floating-point number between 0 and 1
random.randint(1, 10) #random int between 1 and 10
random.choice(['Life of Brian', 'Holy Grail', 'Meaning of Life'])

# shuffle items randomly
suits = ['hearts', 'clubs', 'diamonds', 'spades']
random.shuffle(suits)
suits # => ['spades', 'hearts', 'diamonds', 'clubs']
Casting
str(x)   #converts x to string
list(x)  #converts x to a list
int(x)   #converts x to a integer number
float(x) #converts x to a float number
Other built-in functions
ord(S) #converts from one-character string to integer character code
chr(I) #converts from integer code back to char (ascii only)

min(L) #returns the minimum value in L
max(L) #returns the maximum value in L
sum(L) #returns the sum of the values in L
abs(n) #returns the absolute value of n
pow(n1,n) #returns the exponentiation (power) of n1 on n
round(n1,n) #returns the n1 number rounded to n digits

range(n1,n2,n) #numbers from n1 to n2 in steps of n
type(x) #returns the type of x (string, float…)
help(s) #prints help about x
Regex
import re

re.match(r'^[aeiou]', str)
re.sub(r'^[aeiou]', '?', str)
re.sub(r'(xyz)', r'\1', str)

expr = re.compile(r'^...$')
expr.match(...)
expr.sub(...)

Classes and inheritance

Classes
# We use the "class" statement to create a class
class Human:

    # A class attribute. It is shared by all instances of this class
    species = "H. sapiens"

    # Basic initializer, this is called when this class is instantiated.
    # Note that the double leading and trailing underscores denote objects
    # or attributes that are used by Python but that live in user-controlled
    # namespaces. Methods(or objects or attributes) like: __init__, __str__,
    # __repr__ etc. are called special methods (or sometimes called dunder methods)
    # You should not invent such names on your own.
    def __init__(self, name):
        # Assign the argument to the instance's name attribute
        self.name = name

        # Initialize property
        self._age = 0

    # An instance method. All methods take "self" as the first argument
    def say(self, msg):
        print("{name}: {message}".format(name=self.name, message=msg))

    # Another instance method
    def sing(self):
        return 'microphone check... one two... one two...'

    # A class method is shared among all instances
    # They are called with the calling class as the first argument
    @classmethod
    def get_species(cls):
        return cls.species

    # A static method is called without a class or instance reference
    @staticmethod
    def grunt():
        return "*grunt*"

    # A property is just like a getter.
    # It turns the method age() into an read-only attribute of the same name.
    # There's no need to write trivial getters and setters in Python, though.
    @property
    def age(self):
        return self._age

    # This allows the property to be set
    @age.setter
    def age(self, age):
        self._age = age

    # This allows the property to be deleted
    @age.deleter
    def age(self):
        del self._age


# When a Python interpreter reads a source file it executes all its code.
# This __name__ check makes sure this code block is only executed when this
# module is the main program.
if __name__ == '__main__':
    # Instantiate a class
    i = Human(name="Ian")
    i.say("hi")                     # "Ian: hi"
    j = Human("Joel")
    j.say("hello")                  # "Joel: hello"
    # i and j are instances of type Human, or in other words: they are Human objects

    # Call our class method
    i.say(i.get_species())          # "Ian: H. sapiens"
    # Change the shared attribute
    Human.species = "H. neanderthalensis"
    i.say(i.get_species())          # => "Ian: H. neanderthalensis"
    j.say(j.get_species())          # => "Joel: H. neanderthalensis"

    # Call the static method
    print(Human.grunt())            # => "*grunt*"

    # Cannot call static method with instance of object
    # because i.grunt() will automatically put "self" (the object i) as an argument
    print(i.grunt())                # => TypeError: grunt() takes 0 positional arguments but 1 was given

    # Update the property for this instance
    i.age = 42
    # Get the property
    i.say(i.age)                    # => "Ian: 42"
    j.say(j.age)                    # => "Joel: 0"
    # Delete the property
    del i.age
    # i.age                         # => this would raise an AttributeError
Inheritance
# Inheritance allows new child classes to be defined that inherit methods and
# variables from their parent class.

# Using the Human class defined above as the base or parent class, we can
# define a child class, Superhero, which inherits the class variables like
# "species", "name", and "age", as well as methods, like "sing" and "grunt"
# from the Human class, but can also have its own unique properties.

# To take advantage of modularization by file you could place the classes above in their own files,
# say, human.py

# To import functions from other files use the following format
# from "filename-without-extension" import "function-or-class"

from human import Human


# Specify the parent class(es) as parameters to the class definition
class Superhero(Human):

    # If the child class should inherit all of the parent's definitions without
    # any modifications, you can just use the "pass" keyword (and nothing else)
    # but in this case it is commented out to allow for a unique child class:
    # pass

    # Child classes can override their parents' attributes
    species = 'Superhuman'

    # Children automatically inherit their parent class's constructor including
    # its arguments, but can also define additional arguments or definitions
    # and override its methods such as the class constructor.
    # This constructor inherits the "name" argument from the "Human" class and
    # adds the "superpower" and "movie" arguments:
    def __init__(self, name, movie=False,
                 superpowers=["super strength", "bulletproofing"]):

        # add additional class attributes:
        self.fictional = True
        self.movie = movie
        self.superpowers = superpowers

        # The "super" function lets you access the parent class's methods
        # that are overridden by the child, in this case, the __init__ method.
        # This calls the parent class constructor:
        super().__init__(name)

    # override the sing method
    def sing(self):
        return 'Dun, dun, DUN!'

    # add an additional instance method
    def boast(self):
        for power in self.superpowers:
            print("I wield the power of {pow}!".format(pow=power))


if __name__ == '__main__':
    sup = Superhero(name="Tick")

    # Instance type checks
    if isinstance(sup, Human):
        print('I am human')
    if type(sup) is Superhero:
        print('I am a superhero')

    # Get the Method Resolution search Order used by both getattr() and super()
    # This attribute is dynamic and can be updated
    print(Superhero.__mro__)    # => (<class '__main__.Superhero'>,
                                # => <class 'human.Human'>, <class 'object'>)

    # Calls parent method but uses its own class attribute
    print(sup.get_species())    # => Superhuman

    # Calls overridden method
    print(sup.sing())           # => Dun, dun, DUN!

    # Calls method from Human
    sup.say('Spoon')            # => Tick: Spoon

    # Call method that exists only in Superhero
    sup.boast()                 # => I wield the power of super strength!
                                # => I wield the power of bulletproofing!

    # Inherited class attribute
    sup.age = 31
    print(sup.age)              # => 31

    # Attribute that only exists within Superhero
    print('Am I Oscar eligible? ' + str(sup.movie))
Multiple inheritance
# Another class definition
# bat.py
class Bat:

    species = 'Baty'

    def __init__(self, can_fly=True):
        self.fly = can_fly

    # This class also has a say method
    def say(self, msg):
        msg = '... ... ...'
        return msg

    # And its own method as well
    def sonar(self):
        return '))) ... ((('

if __name__ == '__main__':
    b = Bat()
    print(b.say('hello'))
    print(b.fly)


# And yet another class definition that inherits from Superhero and Bat
# superhero.py
from superhero import Superhero
from bat import Bat

# Define Batman as a child that inherits from both Superhero and Bat
class Batman(Superhero, Bat):

    def __init__(self, *args, **kwargs):
        # Typically to inherit attributes you have to call super:
        # super(Batman, self).__init__(*args, **kwargs)
        # However we are dealing with multiple inheritance here, and super()
        # only works with the next base class in the MRO list.
        # So instead we explicitly call __init__ for all ancestors.
        # The use of *args and **kwargs allows for a clean way to pass arguments,
        # with each parent "peeling a layer of the onion".
        Superhero.__init__(self, 'anonymous', movie=True,
                           superpowers=['Wealthy'], *args, **kwargs)
        Bat.__init__(self, *args, can_fly=False, **kwargs)
        # override the value for the name attribute
        self.name = 'Sad Affleck'

    def sing(self):
        return 'nan nan nan nan nan batman!'


if __name__ == '__main__':
    sup = Batman()

    # Get the Method Resolution search Order used by both getattr() and super().
    # This attribute is dynamic and can be updated
    print(Batman.__mro__)       # => (<class '__main__.Batman'>,
                                # => <class 'superhero.Superhero'>,
                                # => <class 'human.Human'>,
                                # => <class 'bat.Bat'>, <class 'object'>)

    # Calls parent method but uses its own class attribute
    print(sup.get_species())    # => Superhuman

    # Calls overridden method
    print(sup.sing())           # => nan nan nan nan nan batman!

    # Calls method from Human, because inheritance order matters
    sup.say('I agree')          # => Sad Affleck: I agree

    # Call method that exists only in 2nd ancestor
    print(sup.sonar())          # => ))) ... (((

    # Inherited class attribute
    sup.age = 100
    print(sup.age)              # => 100

    # Inherited attribute from 2nd ancestor whose default value was overridden.
    print('Can I fly? ' + str(sup.fly)) # => Can I fly? False

Error handling

# Handle exceptions with a try/except block
try:
    # Use "raise" to raise an error
    raise IndexError("This is an index error")
except IndexError as e:
    # Pass is just a no-op. Usually you would do recovery here.
    pass
except (TypeError, NameError):
    # Multiple exceptions can be handled together, if required.
    pass
else: # Optional clause to the try/except block. Must follow all except blocks
    # Runs only if the code in try raises no exceptions
    print("All good!")
finally: #  Execute under all circumstances
    print("We can clean up resources here")

# Instead of try/finally to cleanup resources you can use a with statement
with open("myfile.txt") as f:
    for line in f:
        print(line)

Common mistakes

Silly things
  • Write return immediately after defining function.

  • Misspellings often dont generate errors. Combat them by using pylint and writing tests regularly.

  • Beware of mixing up def and class

Style
  • Hungarian notation is blnNO.

  • Listen to PEP8.

  • Use lambdas sparingly.

  • Complex comprehension should go into a function.

Structure
Pathological If/Elif Blocks (anti-pattern)

If we really do need to manage many special cases, we can employ the Strategy pattern:

def strategy1():
    ...
def strategy2():
    ...
strategies = {
    'condition1': strategy1,
    'condition2': strategy2,
    ...
}
def do_awesome_stuff():
    which_one = ...
    strategy = strategies[which_one]
    strategy()
    ...

We start by extracting the contents of our if / elif / else structure into separate functions with identical interfaces. Then we can create a dictionary to map conditions to those strategy functions. The dic‐ tionary key doesn’t have to be a string. It can be anything hashable, so tuples and frozensets can be quite effective if we need richer con‐ ditions. Finally, our original function determines which key to use, plucks the appropriate strategy function from our dictionary, and invokes it. Our original function is now much, much simpler to understand, as are each of the strategies, and writing tests for each of the now- isolated strategies is straightforward. However, figuring out what value to use for that dictionary key can sometimes be complicated. If it takes 200 lines to determine what key to use, is this really much of a victory?

If that’s the case, consider externalizing it entirely, and let the strategy be chosen by the caller, who may in fact know better than we do about whatever those factors are. The strategy is invoked as a call‐back:

def do_awesome_stuff(strategy):
    ...
    strategy()
    ...

result = do_awesome_stuff(strategy1)

From there it’s not too far of a jump into dependency injection, where our code is provided with what it needs, rather than having to be smart enough to ask for it on its own:

class Foo(object):
    def __init__(self, strategy):
        self.strategy = strategy

    def do_awesome_stuff(self):
        ...

    self.strategy()
        ...

foo = Foo(strategy2)
foo.do_awesome_stuff()
Unnecessary Getters and Setters

Make most attributes public, and use properties to protect any special snowflakes that need extra care and feeding.

Don’t do:

class InviteEvent(object):
    ...
    def getEventNumber(self):
        return self._intEventNumber

    def setEventNumber(self, x):
        self._intEventNumber = int(x)
    ...

event.setEventNumber(10)
print event.getEventNumber()

Do this instead:

class InviteEvent(object):
    ...
    @property
    def event_number(self):
        return self._event_number

    @event_number.setter
    def _set_event_number(self, x):
        self._event_number = int(x)

    @event_number.deleter
    def _delete_event_number(self):
        self._event_number = None

event.event_number = 10
print event.event_number
Getting Wrapped Up in Decorators

A decorator is a function (or, more generally, a callable) that returns a function, which replaces the function being decorated.

Did you want to test the original function in isolation? Too bad— that function is effectively gone. Your test has no choice but to exercise the final, multilayered Frankenstein function.

Make the decorated method as simple and devoid of logic as possible, pushing all of its smarts down into a deeper layer of abstraction that can be tested in isolation.

Breaking the law of Demeter

The Law of Demeter (also known as the principle of least knowledge) tells us that our code should only interact with the things that it knows about, and not reach deeply into nested attributes, across friends of friends, and into strangers.

Don’t do this:

gvars.objSession.objCustomer.objMemberStatus.isPAID()
if gvars.dctEnv['session'].getCustomer().isSignedIn():
current_url = self.objSession._getCurrentURL() #underscore!
return event._objGuestList.getGuestList()[0].getEventSequence()
Overusing Private Attributes

Since single-underscore names aren’t mangled (unlike double underscore names), they can be more conveniently used if you absolutely must break the Law of Demeter.

When “outside” code wants access to the internals of a class, those “internal” attributes probably shouldn’t be private at all; rather, this is a clear signal that those attributes should be public. The code is telling us that it’s time to refactor!

God Objects and God Methods

When a class or method has accumulated too much knowledge or too many responsibilities, its role in the system becomes practically godlike: it has become all-encompassing, all- seeing, and all-doing, and many other entities will end up being tightly coupled to it in order to get anything done.

Unix command-line kung fu to identify potential sources of godlike trouble:

$ find . -name "*.py" -exec wc -l {} \; | sort -r
$ grep "^class " bigmodule.py | wc -l
$ grep "\sdef " bigmodule.py | wc -l

If you find these kinds of abominations in your code, it’s a sign that it’s time to take a deep breath and refactor them. Favor small functions and small classes that have as few responsibilities as possible, and strive to do as little work as possible in the init so that your classes are easy to instantiate, with no weird side effects, and your tests can be easy and lightweight. You want to break up these wanna-be gods before they get out of hand.

Global State

Avoid global state as much as humanly possible. Resist its siren lure, reject its convenience, refuse the temptation, no matter how much your colleagues think they want it. In the long run, they’ll thank you for not having to maintain such monstrosities.

Surprises
Importing Everything

Don’t. Listen to pep8 and avoid wildcard imports.

Overbroadly Silencing Exceptions

Diaper Pattern

Never catch all errors:

try:
    do_something()
except:
    pass

In most cases, the best choice is to catch a more specific exception. Something like this:

try:
    do_something()
# Catch some very specific exception - KeyError, ValueError, etc.
except ValueError:
    pass

If some code path simply must broadly catch all exceptions - for example, the top-level loop for some long-running persistent process - then each caught exception must write the full stack trace to a log or file, along with a timestamp.

import logging

def get_number():
    return int('foo')

try:
    x = get_number()
except Exception as ex:
    logging.exception('Caught an error')
Reinventing the Wheel
  • Look at the standard library and PyPI to see if someone has already solved your problem. If whatever problem it is isn’t your core domain, your home-grown implementation is probably going to be worse.

  • If you decide to replace one solution in favor of another, see it through; update the codebase to your new standard so that you don’t have a dozen ways to do essentially the same thing.

  • Establish and enforce standards so that surprises—both during development and at runtime—are minimized.

Mutable Keyword Argument Defaults

Python doesn’t give us a new list every time the function gets called; it creates that empty list when the function is defined during the import of its module. And so the following is incredibly dangerous and hides a bug:

def set_reminders(self, event, reminders=[]):

When the default value really does need to be mutable, we set it to None in the function definition, and then immediately give it a reasonable default inside the function itself:

def set_reminders(self, event, reminders=None):
    reminders = reminders or []
    # or, if there are valid falsey inputs
    # that we'd like to preserve:
    reminders = [] if reminders is None else reminders
    ...

This way we’re guaranteed to start with a fresh instance each time the function is called, and data won’t leak between invocations.

Overeager code
  • Don’t do expensive things at import.

  • Don’t couple to resources that might not be available at import.

  • Don’t put anything into an init.py that could jeopardize the import.

  • Don’t do too much when an object is instantiated

  • Beware of circular imports.

Poisoning Persistent State

Any time you’re messing with the contents of a module, or of a class definition, or of anything else that persists outside the scope of a function call, you have an opportunity to shoot yourself in the foot. Proceed with caution when you find yourself writing code like this, and make good use of logging to verify that your assumptions hold.

Assuming Logging Is Unnecessary
  • “This code is too simple to need logging.”

  • “The service I’m integrating with will always work.”

  • “I’ll add logging later.” No. Logging, now.

Log at Boundaries

That can be when entering or leaving a method, when branching ( if / elif/ else ) or looping ( for, while ), when there might be errors ( try / except / finally ), or before and after calling some external service. The type of boundary will guide your choice of log level; for example, debug is best in branching and looping situations, where info makes more sense when entering or leaving larger blocks.

Log Actions, Motives, and Results

Don’t just log what you’re doing, but why, and what happened. This can include actions you’re about to take, decisions made and the information used to make them, errors and exceptions, and things like the URL of a service you’re calling, the data you’re sending to it, and what it returned.

Log Mindfully

  • Unless you have a fancy aggregation tool, like Splunk or Loggly, a single application (or website) should share a single log file.

  • Choose an appropriate level when logging messages. Take a moment to really think about whether a message is for debugging, general information, a caution, or an error.

  • Be mindful of unsanitized user input, of users’ person‐ ally identifiable information, and especially health and payment data.

Assuming tests are unnecessary

As long as our code is syntactically reasonable, Python will cheer‐ fully do its best to execute it, even if that means we’ve forgotten to return a value, gotten our types mismatched, mixed up a sign in some tricky math, used the wrong variable or misspelled a variable name, or committed any number of other common programmer errors. When we have unit tests, we learn about our errors up front, as we make them, rather than during integration—or worse, production—where it’s much more expensive to resolve them. As an added bonus, when your code can be easily tested, it is more likely to be better structured and thus cleaner and more maintainable.

So go make friends with unittest , Pytest, or Nose, and explore what the Mock library can do to help you isolate components from one another. Get comfortable with testing, practice it until it becomes like a reflex. Utilize logging.

Testing now will help prevent weird surprises later.

Gotchas
Common Gotchas

For the most part, Python aims to be a clean and consistent language that avoids surprises. However, there are a few cases that can be confusing to newcomers.

Some of these cases are intentional but can be potentially surprising. Some could arguably be considered language warts. In general, what follows is a collection of potentially tricky behavior that might seem strange at first glance, but is generally sensible once you’re aware of the underlying cause for the surprise.

Mutable Default Arguments

Seemingly the most common surprise new Python programmers encounter is Python’s treatment of mutable default arguments in function definitions.

What You Wrote

def append_to(element, to=[]):
    to.append(element)
    return to

What You Might Have Expected to Happen

my_list = append_to(12)
print(my_list)

my_other_list = append_to(42)
print(my_other_list)

A new list is created each time the function is called if a second argument isn’t provided, so that the output is:

    [12]
    [42]

What Does Happen

[12]
[12, 42]

A new list is created once when the function is defined, and the same list is used in each successive call.

Python’s default arguments are evaluated once when the function is defined, not each time the function is called (like it is in say, Ruby). This means that if you use a mutable default argument and mutate it, you will and have mutated that object for all future calls to the function as well.

What You Should Do Instead

Create a new object each time the function is called, by using a default arg to signal that no argument was provided (None is often a good choice).

def append_to(element, to=None):
    if to is None:
        to = []
    to.append(element)
    return to

Do not forget, you are passing a list object as the second argument.

When the Gotcha Isn’t a Gotcha

Sometimes you can specifically "exploit" (read: use as intended) this behavior to maintain state between calls of a function. This is often done when writing a caching function.

Late Binding Closures

Another common source of confusion is the way Python binds its variables in closures (or in the surrounding global scope).

What You Wrote

def create_multipliers():
    return [lambda x : i * x for i in range(5)]

What You Might Have Expected to Happen

for multiplier in create_multipliers():
    print(multiplier(2))

A list containing five functions that each have their own closed-over i variable that multiplies their argument, producing:

0
2
4
6
8

What Does Happen

8
8
8
8
8

Five functions are created; instead all of them just multiply x by 4.

Python’s closures are late binding. This means that the values of variables used in closures are looked up at the time the inner function is called.

Here, whenever any of the returned functions are called, the value of i is looked up in the surrounding scope at call time. By then, the loop has completed and i is left with its final value of 4.

What’s particularly nasty about this gotcha is the seemingly prevalent misinformation that this has something to do with lambdas in Python. Functions created with a lambda expression are in no way special, and in fact the same exact behavior is exhibited by just using an ordinary def:

def create_multipliers():
    multipliers = []

    for i in range(5):
        def multiplier(x):
            return i * x
        multipliers.append(multiplier)

    return multipliers

What You Should Do Instead

The most general solution is arguably a bit of a hack. Due to Python’s aforementioned behavior concerning evaluating default arguments to functions (see Mutable Default Arguments), you can create a closure that binds immediately to its arguments by using a default arg like so:

def create_multipliers():
    return [lambda x, i=i : i * x for i in range(5)]

Alternatively, you can use the functools.partial function:

from functools import partial
from operator import mul

def create_multipliers():
    return [partial(mul, i) for i in range(5)]

When the Gotcha Isn’t a Gotcha Sometimes you want your closures to behave this way. Late binding is good in lots of situations. Looping to create unique functions is unfortunately a case where they can cause hiccups.

SQL cheatsheet and notes

Select

--  filter your columns
SELECT col1, col2, col3, ... FROM table1

--  filter the rows
WHERE col4 = 1 AND col5 = 2

-- aggregate the data
GROUP by ...

-- limit aggregated data
HAVING count(*) > 1

-- order of the results
ORDER BY col2
Useful keywords for SELECTS:
  • DISTINCT - return unique results

  • BETWEEN a AND b - limit the range, the values can be numbers, text, or dates

  • LIKE - pattern search within the column text

  • IN (a, b, c) - check if the value is contained among given.

Data Modification

-- update specific data with the WHERE clause
UPDATE table1 SET col1 = 1 WHERE col2 = 2

-- insert values manually
INSERT INTO table1 (ID, FIRST_NAME, LAST_NAME) VALUES (1, ‘Rebel’, ‘Labs’);

-- or by using the results of a query
INSERT INTO table1 (ID,  FIRST_NAME,  LAST_NAME)
   SELECT id, last_name, first_name FROM table2

View

A VIEW is a virtual table, which is a result of a query. They can be used to create virtual tables of complex queries.

CREATE VIEW view1 AS
 SELECT col1, col2
 FROM  table1
 WHERE  ..

Join

SELECT * FROM A INNER JOIN B

┌────────┐
│ A  ┌───┼────┐
│    │ ∩ │    │
└────┼───┘  B │
     └────────┘

Inner join: ∩  - fetch the results that exist in both tables
Left outer join: A + ∩ - all rows from table A, even if they dont exist in table B
Right outer join: ∩ + B - all rows from table B, even if they dont exist in table A
Full outer join: A + ∩ + B

You can use subqueries instead of JOINs:

SELECT col1, col2 FROM table1 WHERE id IN
  (SELECT t1_id FROM table2 WHERE date > CURRENT_TIMESTAMP)

Utility functions

-- convert strings to dates:
TO_DATE (Oracle, PostgreSQL), STR_TO_DATE (MySQL)

-- return the first non-NULL argument:
COALESCE (col1, col2, “default value”)

-- return current time:
CURRENT_TIMESTAMP

-- compute set operations on two result sets
SELECT col1, col2 FROM table1
UNION / EXCEPT / INTERSECT
SELECT col3, col4 FROM table2;
  • Union - returns data from both queries

  • Except - rows from the first query that are not present in the second query

  • Intersect - rows that are returned from both queries

Reporting

Use aggregation functions
  • COUNT - return the number of rows

  • SUM - cumulate the values

  • AVG - return the average for the group

  • MIN / MAX - smallest / largest value

Algorithms

  • Its input is a sorted list of elements,

  • If an element you’re looking for is in that list, binary search returns the position where it’s located. Otherwise, binary search returns null.

  • In general, for any list of n, binary search will take log2(n) steps (log time) to run in the worst case, whereas simple search will take n steps (linear time).

  • For a list of 1,024 elements, log 1,024 = 10, because 2^10 == 1,024. So for a list of 1,024 numbers, you’d have to check 10 numbers at most.

# The binary_search function takes a sorted array and an item. If the
# item is in the array, the function returns its position.
def binary_search(lst, item):
    # keep track of where we are in the lsit
    low = 0
    high = len(lst) -1

    # while we arent down to one element...
    while low <= high:
        # check middle element
        mid = (low + high) // 2
        guess = lst[mid]

        if guess == item: # found the item
            return mid
        elif guess > item: # the guess was too high
            high = mid -1
        elif guess < item: # the guess was too low
            low = mid + 1

    return None # we didnt find the item

#lets test it
test = [1,2,3,4,5,6,7,8,9,10]
print(binary_search(test, 3))
print(binary_search(test, 8))
Big O notation

Run time of algorithms is expressed in Big O notation.

| num elements | linear search | logarithmic search |
| ------------ | ------------- | ------------------ |
| 100          | 100ms         | 7ms                |
| 10000        | 10seconds     | 14ms               |
| 1000000000   | 11days        | 32ms               |

Big O establishes a worst-case run time. Along with the worst-case run time, it’s also important to look at the average-case run time.

Common Big O run times sorted from fastest to slowest - O(log n), also known as log time. Binary search. - O(n), also known as linear time. Simple search. - O(n * log n) A fast sorting algorithm, like quicksort - O(n^2 ) A slow sorting algorithm, like selection sort - O(n!) A really slow algorithm, like the traveling salesperson

Algorithm speed isn’t measured in seconds, but in growth of the number of operations. Instead, we talk about how quickly the run time of an algorithm increases as the size of the input increases.

Selection Sort

Arrays and linked lists

Using an array means all your tasks are stored contiguously (right next to each other) in memory. With linked lists, your items can be anywhere in memory since each item stores the address of the next item in the list.

Linked lists are great if you’re going to read all the items one at a time: you can read one item, follow the address to the next item, and so on. But if you’re going to keep jumping around, linked lists are terrible. Arrays are different. You know the address for every item in your array.

The position of an element is called its index.

Run times for common operations on arrays and lists:

| operation | arrays | lists |
| --------- | ------ | ----- |
| reading   | O(1)   | O(n)  |
| insertion | O(n)   | O(1)  |
| deletion  | O(n)   | O(1)  |

O(n) = linear time, O(1) = constant time

Arrays see a lot of use because they allow random access. Here are two different types of access: random access and sequential access. Sequential access means reading the elements one by one, starting at the first element. Linked lists can only do sequential access. If you want to read the 10th element of a linked list, you have to read the first 9 elements and follow the links to the 10th element. Random access means you can jump directly to the 10th element.

Selection sort

In computer science, selection sort is a sorting algorithm, specifically an in-place comparison sort. It has O(n2) time complexity, making it inefficient on large lists, and generally performs worse than the similar insertion sort. Selection sort is noted for its simplicity, and it has performance advantages over more complicated algorithms in certain situations, particularly where auxiliary memory is limited.

| Class                       | Sorting algorithm             |
| --------------------------- | ----------------------------- |
| Data structure              | Array                         |
| Worst-case performance      | О(n2) comparisons, О(n) swaps |
| Best-case performance       | О(n2) comparisons, О(n) swaps |
| Average performance         | О(n2) comparisons, О(n) swaps |
| Worst-case space complexity | O(1) auxiliary                |
Algorithm

The algorithm divides the input list into two parts: the sublist of items already sorted, which is built up from left to right at the front (left) of the list, and the sublist of items remaining to be sorted that occupy the rest of the list. Initially, the sorted sublist is empty and the unsorted sublist is the entire input list. The algorithm proceeds by finding the smallest (or largest, depending on sorting order) element in the unsorted sublist, exchanging (swapping) it with the leftmost unsorted element (putting it in sorted order), and moving the sublist boundaries one element to the right.

Selection sort Implementation
def findSmallest(arr):
    smallest = arr[0]
    smallest_index = 0

    for i in range(1, len(arr)):
        if arr[i] < smallest:
            smallest_index = i
    return smallest_index

def selectionSort(arr):
    newArr = []
    for i in range(len(arr)):
        smallest = findSmallest(arr)
        newArr.append(arr.pop(smallest))
    return newArr

print(selectionSort([5,4,6,2,10,13,9]))

Recap: Arrays allow fast reads. Linked lists allow fast inserts and deletes. All elements in the array should be the same type (all ints, all doubles, and so on).

Recursion

hat’s why every recursive function has two parts: the base case, and the recursive case. he recursive case is when the function calls itself. he base case is when the function doesn’t call itself again …​ so it doesn’t go into an infinite loop.

# countdown using recursion
def countdown(i):
    print(i)
    if i <= 0: # Base case
        return
    else: # Recursive case
        countdown(i-1)
The stack

In computer science, a stack is an abstract data type that serves as a collection of elements, with two principal operations: - push, which adds an element to the collection, and - pop, which removes the most recently added element that was not yet removed.

The call stack

In computer science, a call stack is a stack data structure that stores information about the active subroutines of a computer program. This kind of stack is also known as an execution stack, program stack, control stack, run-time stack, or machine stack, and is often shortened to just "the stack"

All function calls go onto the call stack. he call stack can get very large, which takes up a lot of memory.

Quicksort

Divide & conquer

Divide and conquer is an algorithm design paradigm based on multi-branched recursion. A divide and conquer algorithm works by recursively breaking down a problem into two or more sub-problems of the same or related type, until these become simple enough to be solved directly. The solutions to the sub-problems are then combined to give a solution to the original problem.

This divide and conquer technique is the basis of efficient algorithms for all kinds of problems, such as sorting (e.g., quicksort, merge sort), multiplying large numbers (e.g. the Karatsuba algorithm), finding the closest pair of points, syntactic analysis (e.g., top-down parsers), and computing the discrete Fourier transform (FFTs).

# sum of numbers in array using loop
def sum(arr):
    total = 0
    for i in arr:
        total += i
    return total

# sum of numbers in array using recursion
def sum2(arr):
    if arr == []:
        return 0
    return arr[0] + sum(arr[1:])

# recursive function to count the number of items in a list
def count(lst):
    if lst == []:
        return 0
    return 1 + count(lst[1:])

# Find the maximum number in a list using recursion.
def find_max(lst):
    if len(lst) == 0:
        return None
    if len(lst) == 1:
        return lst[0]
    else:
        sub_max = find_max(lst[1:])
        return lst[0] if lst[0] > sub_max else sub_max

print(sum([1,2,3,4,5]))
print(sum2([1,2,3,4,5]))
print(count([1,2,3,'a',4,'b',5]))
print(find_max([1,2,3,4,5]))
Quicksort

Quicksort (sometimes called partition-exchange sort) is an efficient sorting algorithm, serving as a systematic method for placing the elements of an array in order.

Quicksort is a comparison sort, meaning that it can sort items of any type for which a "less-than" relation (formally, a total order) is defined. In efficient implementations it is not a stable sort, meaning that the relative order of equal sort items is not preserved. Quicksort can operate in-place on an array, requiring small additional amounts of memory to perform the sorting. It is very similar to selection sort, except that it does not always choose worst-case partition.

The C standard library has a function called qsort, which is its implementation of quicksort.

| Class                       | Sorting algorithm |
| --------------------------- | ----------------- |
| Worst-case performance      | O(n2)             |
| Best-case performance       | O(n log n)        |
| Average performance         | O(n log n)        |
| Worst-case space complexity | O(n) / O(log n)   |
Algorithm

Quicksort is a divide and conquer algorithm. Quicksort first divides a large array into two smaller sub-arrays: the low elements and the high elements. Quicksort can then recursively sort the sub-arrays.

The steps are: - Pick an element, called a pivot, from the array. - Partitioning: reorder the array so that all elements with values less than the pivot come before the pivot, while all elements with values greater than the pivot come after it (equal values can go either way). After this partitioning, the pivot is in its final position. This is called the partition operation. - Recursively apply the above steps to the sub-array of elements with smaller values and separately to the sub-array of elements with greater values.

The base case of the recursion is arrays of size zero or one, which are in order by definition, so they never need to be sorted.

Quicksort Implementation
def quicksort(array):
    if len(array) < 2:
        return array # Base case: arrays with 0 or 1 element are already "sorted."
    else:
        pivot = array[0] # Recursive case
        less = [i for i in array[1:] if i <= pivot] # Sub-array of all the elements less than the pivot
        greater = [i for i in array[1:] if i > pivot] # Sub-array of all the elements greater than the pivot
    return quicksort(less) + [pivot] + quicksort(greater)

print(quicksort([10, 5, 2, 3]))
Merge sort

Merge sort (also commonly spelled mergesort) is an efficient, general-purpose, comparison-based sorting algorithm. Most implementations produce a stable sort, which means that the implementation preserves the input order of equal elements in the sorted output. Merge sort is a divide and conquer algorithm

| Class                       | Sorting algorithm                                                |
| --------------------------- | ---------------------------------------------------------------- |
| Data structure              | Array                                                            |
| Worst-case performance      | O(n log n)                                                       |
| Best-case performance       | O(n log n) typical, O(n) natural variant                         |
| Average performance         | O(n log n)                                                       |
| Worst-case space complexity | О(n) total with O(n) auxiliary, O(1) auxiliary with linked lists |
Algorithm

Conceptually, a merge sort works as follows: - Divide the unsorted list into n sublists, each containing 1 element (a list of 1 element is considered sorted). - Repeatedly merge sublists to produce new sorted sublists until there is only 1 sublist remaining. This will be the sorted list.

Merge sort Implementation
def mergeSort(lst):
    if len(lst)>1:
        mid = len(lst)//2
        lefthalf = lst[:mid]
        righthalf = lst[mid:]

        mergeSort(lefthalf)
        mergeSort(righthalf)

        i = 0, j = 0, k = 0

        while i < len(lefthalf) and j < len(righthalf):
            if lefthalf[i] < righthalf[j]:
                lst[k] = lefthalf[i]
                i = i + 1
            else:
                lst[k] = righthalf[j]
                j = j + 1
            k = k + 1

        while i < len(lefthalf):
            lst[k] = lefthalf[i]
            i = i + 1
            k = k + 1

        while j < len(righthalf):
            lst[k] = righthalf[j]
            j = j + 1
            k = k + 1

lst = [54,26,93,17,77,31,44,55,20]
print(mergeSort(lst))

Hash Tables

Also known as hash maps, maps, dictionaries, and associative arrays.

In computing, a hash table (hash map) is a data structure that implements an associative array abstract data type, a structure that can map keys to values. A hash table uses a hash function to compute an index into an array of buckets or slots, from which the desired value can be found.

Ideally, the hash function will assign each key to a unique bucket, but most hash table designs employ an imperfect hash function, which might cause hash collisions where the hash function generates the same index for more than one key. Such collisions must be accommodated in some way.

| Algorithm | Average | Worst case |
| --------- | ------- | ---------- |
| Space     | O(n)    | O(n)       |
| Search    | O(1)    | O(n)       |
| Insert    | O(1)    | O(n)       |
| Delete    | O(1)    | O(n)       |

Python has hash tables; they’re called dictionaries. You can make a new hash table using the dict function: book = dict()

>>> book["apple"] = 0.67 # an apple costs 67 cents
>>> book["milk"] = 1.49  # Milk costs $1.49.
>>> book["avocado"] = 1.49
>>> print book
{'avocado': 1.49, 'apple': 0.67, 'milk': 1.49}

>>> print book["avocado"]
1.49

In C, unordered associative containers are a group of class templates in the C Standard Librarythat implement hash table variants. Being templates, they can be used to store arbitraryelements, such as integers or custom classes. The following containers are defined in thecurrent revision of the C++ standard: unordered_set, unordered_map, unordered_multiset, unordered_multimap. Each of these containers differ only on constraints placed on their elements.

The unordered associative containers are similar to the associative containers in the C++ Standard Library but have different constraints. As their name implies, the elements in the unordered associative containers are not ordered. This is due to the use of hashing to store objects. The containers can still be iterated through like a regular associative container.

#include <iostream>
#include <string>
#include <unordered_map>

int main(){
    std::unordered_map<std::string, int> months;
    months["april"] = 30;
    months["september"] = 30;
    std::cout << "september -> " << months["september"] << std::endl;
    std::cout << "april     -> " << months["april"] << std::endl;
    return 0;
}
Recap

To recap, hashes are good for - Modeling relationships from one thing to another thing - Filtering out duplicates - Caching/memorizing data instead of making your server do work

Graph

In computer science, a graph is an abstract data type that is meant to implement the undirected graph and directed graph concepts from mathematics, specifically the field of graph theory.

A graph data structure consists of a finite (and possibly mutable) set of vertices or nodes or points, together with a set of unordered pairs of these vertices for an undirected graph or a set of ordered pairs for a directed graph. These pairs are known as edges, arcs, or lines for an undirected graph and as arrows, directed edges, directed arcs, or directed lines for a directed graph. The vertices may be part of the graph structure, or may be external entities represented by integer indices or references.

A graph data structure may also associate to each edge some edge value, such as a symbolic label or a numeric attribute (cost, capacity, length, etc.).

Breadth-first search

Breadth-first search (BFS) is an algorithm for traversing or searching tree or graph data structures. It starts at the tree root (or some arbitrary node of a graph, sometimes referred to as a 'search key'), and explores all of the neighbor nodes at the present depth prior to moving on to the nodes at the next depth level.

It uses the opposite strategy as depth-first search, which instead explores the highest-depth nodes first before being forced to backtrack and expand shallower nodes.

Queue

In computer science, a queue is a particular kind of abstract data type or collection in which the entities in the collection are kept in order and the principal (or only) operations on the collection are the addition of entities to the rear terminal position, known as enqueue, and removal of entities from the front terminal position, known as dequeue. This makes the queue a First-In-First-Out (FIFO) data structure. In a FIFO data structure, the first element added to the queue will be the first one to be removed. This is equivalent to the requirement that once a new element is added, all elements that were added before have to be removed before the new element can be removed. Often a peek or front operation is also entered, returning the value of the front element without dequeuing it. A queue is an example of a linear data structure, or more abstractly a sequential collection.

Queues provide services in computer science, transport, and operations research where various entities such as data, objects, persons, or events are stored and held to be processed later. In these contexts, the queue performs the function of a buffer.

Queues are common in computer programs, where they are implemented as data structures coupled with access routines, as an abstract data structure or in object-oriented languages as classes. Common implementations are circular buffers and linked lists.

| Algorithm | Average | Worst case |
| --------- | ------- | ---------- |
| Space     | O(n)    | O(n)       |
| Search    | O(n)    | O(n)       |
| Insert    | O(1)    | O(1)       |
| Delete    | O(1)    | O(1)       |
Implementing the graph
# find mango seller among friends/mutal friends
from collections import deque

graph = {}
graph["you"] = ["alice", "bob", "claire"]
graph["bob"] = ["anuj", "peggy"]
graph["alice"] = ["peggy"]
graph["claire"] = ["thom", "jonny"]
graph["anuj"] = []
graph["peggy"] = []
graph["thom"] = []
graph["jonny"] = []

def person_is_seller(name):
    return name[-1] == 'm'      #if name ends withm m they are mango seller.

def find_mango_seller(name):
    search_queue = deque()          # Creates a new queue
    search_queue += graph["you"]    # Adds all of your neighbors to the search queue
    searched = []                   # keep track of which people you've searched before.
    while search_queue:                 # While the queue isn't empty ...
        person = search_queue.popleft() # ... grabs the first person off the queue
        if person_is_seller(person):    # Checks whether the person is a mango seller
            print(person + " is a mango seller!") # Yes, they're a mango seller.
            return True
        else:
            # No, they aren't. Add all of this person's friends to the search queue.
            search_queue += graph[person]
            searched.append(person) # Mark this person as searched
    return False # If you reached here, no one in the queue was a mango seller.

find_mango_seller("you")

If you search your entire network for a mango seller, that means you’ll follow each edge (remember, an edge is the arrow or connection from one person to another). So the running time is at least O(number of edges). You also keep a queue of every person to search. Adding one person to the queue takes constant time: O(1). Doing this for every person will take O(number of people) total. Breadth-first search takes O(number of people + number of edges), and it’s more commonly written as O(V+E) (V for number of vertices, E for number of edges).

Recap
  • Breadth-first search tells you if there’s a path from A to B.

  • If there’s a path, breadth-first search will ind the shortest path.

  • If you have a problem like "find the shortest X," try modeling your problem as a graph, and use breadth-first search to solve.

  • A directed graph has arrows, and the relationship follows the direction of the arrow (rama → adit means "rama owes adit money").

  • Undirected graphs don’t have arrows, and the relationship goes both ways (ross - rachel means "ross dated rachel and rachel dated ross").

  • Queues are FIFO (First In, First Out).

  • Stacks are LIFO (Last In, First Out).

  • You need to check people in the order they were added to the search list, so the search list needs to be a queue. Otherwise, you won’t get the shortest path.

  • Once you check someone, make sure you don’t check them again. Otherwise, you might end up in an infinite loop.

Dijkstra’s algorithm

Dijkstra’s algorithm is an algorithm for finding the shortest paths between nodes in a graph.

Dijkstra’s algorithm has four steps: 1. Find the cheapest node. This is the node you can get to in the least amount of time. 2. Check whether there’s a cheaper path to the neighbors of this node. If so, update their costs. 3. Repeat until you’ve done this for every node in the graph. 4. Calculate the final path.

When you work with Dijkstra’s algorithm, each edge in the graph has a number associated with it. These are called weights. A graph with weights is called a weighted graph. A graph without weights is called an unweighted graph. To calculate the shortest path in an unweighted graph, use breadth-first search. To calculate the shortest path in a weighted graph, use Dijkstra’s algorithm.

Graphs can also have cycles. It means you can start at a node, travel around, and end up at the same node. An undirected graph with only 2 nodes means that both nodes point to each other. That’s a cycle.

With an undirected graph, each edge adds another cycle. Dijkstra’s algorithm only works with directed acyclic graphs, called DAGs for short.

You can’t use negative-weight edges with Dijkstra’s algorithm. If you want to ind the shortest path in a graph that has negative-weight edges use the Bellman-Ford algorithm.

In some fields, artificial intelligence in particular, Dijkstra’s algorithm or a variant of it is known as uniform cost search and formulated as an instance of the more general idea of best-first search.

Dijkstra’s algorithm Implementation
# the graph hash table
graph = {}
graph["start"] = {}
graph["start"]["a"] = 6
graph["start"]["b"] = 2

graph["a"] = {}
graph["a"]["fin"] = 1

graph["b"] = {}
graph["b"]["a"] = 3
graph["b"]["fin"] = 5

graph["fin"] = {}

# the costs table
infinity = float("inf")
costs = {}
costs["a"] = 6
costs["b"] = 2
costs["fin"] = infinity

# the parents table
parents = {}
parents["a"] = "start"
parents["b"] = "start"
parents["fin"] = None

processed = [] # keep track of processed nodes

# while we have nodes to proces: (grab the node closes to start->
# -> updates costs for its neihgbours -> if any of neighbours costs were updated
# update the parents too -> mark this node as processed ) loop

def find_lowest_cost_node(costs):
    lowest_cost = float("inf")
    lowest_cost_node = None
    for node in costs: # Go through each node.
        cost = costs[node]
        # If it's the lowest cost so far and hasn't been processed yet...
        if cost < lowest_cost and node not in processed:
            # ... set it as the new lowest-cost node.
            lowest_cost = cost
            lowest_cost_node = node
    return lowest_cost_node

# Find the lowest-cost node that you haven't processed yet.
node = find_lowest_cost_node(costs)

# If you've processed all the nodes, this while loop is done.
while node is not None:
    cost = costs[node]
    neighbors = graph[node]
    for n in neighbors.keys():  # Go through all the neighbors of this node.
        new_cost = cost + neighbors[n]
        # If it's cheaper to get to this neighbor by going through this node...
        if costs[n] > new_cost:
            costs[n] = new_cost # ... update the cost for this node.
            parents[n] = node   # This node becomes the new parent for this neighbor.
    processed.append(node) # Mark the node as processed.
    node = find_lowest_cost_node(costs)  # Find the next node to process, and loop.

print("Cost from the start to each node: " + costs)
Recap
  • Breadth-first search is used to calculate the shortest path for an unweighted graph.

  • Dijkstra’s algorithm is used to calculate the shortest path for a weighted graph.

  • Dijkstra’s algorithm works when all the weights are positive.

  • If you have negative weights, use the Bellman-Ford algorithm.

Greedy algorithms

A greedy algorithm is an algorithmic paradigm that follows the problem solving heuristic of making the locally optimal choice at each stage with the intent of finding a global optimum. In many problems, a greedy strategy does not usually produce an optimal solution, but nonetheless a greedy heuristic may yield locally optimal solutions that approximate a globally optimal solution in a reasonable amount of time.

For example, a greedy strategy for the traveling salesman problem (which is of a high computational complexity) is the following heuristic: "At each step of the journey, visit the nearest unvisited city." This heuristic doesn’t intend to find a best solution, but it terminates in a reasonable number of steps; finding an optimal solution to such a complex problem typically requires unreasonably many steps.

In general, greedy algorithms have five components: - A candidate set, from which a solution is created - A selection function, which chooses the best candidate to be added to the solution - A feasibility function, that is used to determine if a candidate can be used to contribute to a solution - An objective function, which assigns a value to a solution, or a partial solution, and - A solution function, which will indicate when we have discovered a complete solution

Approximation algorithms

When calculating the exact solution will take too much time, an approximation algorithm will work. Approximation algorithms are judged by - How fast they are - How close they are to the optimal solution

# covering most states with least radio stations
states_needed = set(["mt", "wa", "or", "id", "nv", "ut", "ca", "az"])

stations = {}
stations["kone"] = set(["id", "nv", "ut"])
stations["ktwo"] = set(["wa", "id", "mt"])
stations["kthree"] = set(["or", "nv", "ca"])
stations["kfour"] = set(["nv", "ut"])
stations["kfive"] = set(["ca", "az"])

final_stations = set()

while states_needed:
    best_station = None
    states_covered = set()
    for station, states in stations.items():
        covered = states_needed & states        # intersection
        if len(covered) > len(states_covered):
            best_station = station
            states_covered = covered
    states_needed -= states_covered
    final_stations.add(best_station)

print(final_stations)
Recap
  • Greedy algorithms optimize locally, hoping to end up with a global optimum.

  • NP-complete problems have no known fast solution.

  • If you have an NP-complete problem, your best bet is to use an approximation algorithm.

  • Greedy algorithms are easy to write and fast to run, so they make good approximation algorithms.

Dynamic programming

In computer science, a problem is said to have optimal substructure if an optimal solution can be constructed from optimal solutions of its subproblems. This property is used to determine the usefulness of dynamic programming and greedy algorithms for a problem.

Dynamic programming is both a mathematical optimization method and a computer programming method. In both contexts it refers to simplifying a complicated problem by breaking it down into simpler sub-problems in a recursive manner. While some decision problems cannot be taken apart this way, decisions that span several points in time do often break apart recursively. Likewise, in computer science, if a problem can be solved optimally by breaking it into sub-problems and then recursively finding the optimal solutions to the sub-problems, then it is said to have optimal substructure.

# Longest common substring
# Alex typed hish. Which word did Alex mean to type: ish or vista?
if word_a[i] == word_b[j]:           # The letters match.
    cell[i][j] = cell[i-1][j-1] + 1
else:                                # The letters don’t match.
    cell[i][j] = 0
# Longest common subsequence
# Suppose Alex accidentally searched for fosh. Which word did he mean:
# fish or fort?
if word_a[i] == word_b[j]:
    # The letters match.
    cell[i][j] = cell[i-1][j-1] + 1
else:
    # The letters don't match.
    cell[i][j] = max(cell[i-1][j], cell[i][j-1])
Recap
  • Dynamic programming is useful when you’re trying to optimize something given a constraint.

  • You can use dynamic programming when the problem can be broken into discrete subproblems.

  • Every dynamic-programming solution involves a grid.

  • he values in the cells are usually what you’re trying to optimize.

  • Each cell is a subproblem, so think about how you can divide your problem into subproblems.

  • here’s no single formula for calculating a dynamic-programming solution.

  • `diff`s use dynamic programming

K-nearest neighbors

In machine learning and statistics, classification is the problem of identifying to which of a set of categories (sub-populations) a new observation belongs, on the basis of a training set of data containing observations (or instances) whose category membership is known. Examples are assigning a given email to the "spam" or "non-spam" class, and assigning a diagnosis to a given patient based on observed characteristics of the patient (gender, blood pressure, presence or absence of certain symptoms, etc.). Classification is an example of pattern recognition.

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors'). More specifically, regression analysis helps one understand how the typical value of the dependent variable (or 'criterion variable') changes when any one of the independent variables is varied, while the other independent variables are held fixed.

In pattern recognition, the k-nearest neighbors algorithm (k-NN) is a non-parametric method used for classification and regression. k-NN is a type of instance-based learning, or lazy learning, where the function is only approximated locally and all computation is deferred until classification. The k-NN algorithm is among the simplest of all machine learning algorithms.

OCR

Optical character recognition (also optical character reader, OCR) is the mechanical or electronic conversion of images of typed, handwritten or printed text into machine-encoded text, whether from a scanned document, a photo of a document, a scene-photo. OCR is a field of research in pattern recognition, artificial intelligence and computer vision.

Feature extraction basic type of core OCR algorithms decomposes glyphs into "features" like lines, closed loops, line direction, and line intersections. The extraction features reduces the dimensionality of the representation and makes the recognition process computationally efficient. These features are compared with an abstract vector-like representation of a character, which might reduce to one or more glyph prototypes. General techniques of feature detection in computer vision are applicable to this type of OCR, which is commonly seen in "intelligent" handwriting recognition and indeed most modern OCR software. Nearest neighbour classifiers such as the k-nearest neighbors algorithm are used to compare image features with stored glyph features and choose the nearest match.

Native Bayes classifier

In machine learning, naive Bayes classifiers are a family of simple "probabilistic classifiers" based on applying Bayes' theorem with strong (naive) independence assumptions between the features.

It is a popular (baseline) method for text categorization, the problem of judging documents as belonging to one category or the other (such as spam or legitimate, sports or politics, etc.) with word frequencies as the features. With appropriate pre-processing, it is competitive in this domain with more advanced methods including support vector machines. It also finds application in automatic medical diagnosis.

Recap
  • KNN is used for classiication and regression and involves looking at the k-nearest neighbors.

  • Classiication = categorization into a group.

  • Regression = predicting a response (like a number).

  • Feature extraction means converting an item (like a fruit or a user) into a list of numbers that can be compared.

  • Picking good features is an important part of a successful KNN algorithm.

Other

Trees

In computer science, a tree is a widely used abstract data type (ADT)—or data structure implementing this ADT—that simulates a hierarchical tree structure, with a root value and subtrees of children with a parent node, represented as a set of linked nodes.

A tree data structure can be defined recursively (locally) as a collection of nodes (starting at a root node), where each node is a data structure consisting of a value, together with a list of references to nodes (the "children"), with the constraints that no reference is duplicated, and none points to the root.

Alternatively, a tree can be defined abstractly as a whole (globally) as an ordered tree, with a value assigned to each node. Both these perspectives are useful: while a tree can be analyzed mathematically as a whole, when actually represented as a data structure it is usually represented and worked with separately by node (rather than as a set of nodes and an adjacency list of edges between nodes, as one may represent a digraph, for instance). For example, looking at a tree as a whole, one can talk about "the parent node" of a given node, but in general as a data structure a given node only contains the list of its children, but does not contain a reference to its parent (if any).

B-trees

B-tree is a self-balancing tree data structure that keeps data sorted and allows searches, sequential access, insertions, and deletions in logarithmic time. The B-tree is a generalization of a binary search tree in that a node can have more than two children. Unlike self-balancing binary search trees, the B-tree is well suited for storage systems that read and write relatively large blocks of data, such as discs. It is commonly used in databases and filesystems.

| Algorithm | Average  | Worst  case |
| --------- | -------- | ----------- |
| Space     | O(n)     | O(n)        |
| Search    | O(log n) | O(log n)    |
| Insert    | O(log n) | O(log n)    |
| Delete    | O(log n) | O(log n)    |
Red-black trees

A red–black tree is a kind of self-balancing binary search tree in computer science. Each node of the binary tree has an extra bit, and that bit is often interpreted as the color (red or black) of the node. These color bits are used to ensure the tree remains approximately balanced during insertions and deletions. The tree does not contain any other data specific to its being a red–black tree so its memory footprint is almost identical to a classic (uncolored) binary search tree. In many cases, the additional bit of information can be stored at no additional memory cost.

| Algorithm | Average  | Worst case |
| --------- | -------- | ---------- |
| Space     | O(n)     | O(n)       |
| Search    | O(log n) | O(log n)   |
| Insert    | O(log n) | O(log n)   |
| Delete    | O(log n) | O(log n)   |
Splay trees

A splay tree is a self-adjusting binary search tree with the additional property that recently accessed elements are quick to access again. It performs basic operations such as insertion, look-up and removal in O(log n) amortized time. For many sequences of non-random operations, splay trees perform better than other search trees, even when the specific pattern of the sequence is unknown.

| Algorithm | Average  | Worst case         |
| --------- | -------- | ------------------ |
| Space     | O(n)     | O(n)               |
| Search    | O(log n) | amortized O(log n) |
| Insert    | O(log n) | amortized O(log n) |
| Delete    | O(log n) | amortized O(log n) |
Heaps

heap is a specialized tree-based data structure that satisfies the heap property: if P is a parent node of C, then the key (the value) of P is either greater than or equal to (in a max heap) or less than or equal to (in a min heap) the key of C. The node at the "top" of the heap (with no parents) is called the root node.

The heap is one maximally efficient implementation of an abstract data type called a priority queue, and in fact priority queues are often referred to as "heaps", regardless of how they may be implemented. A common implementation of a heap is the binary heap, in which the tree is a binary tree.

Python: heapq, C++: <algorithm> → make_heap etc, Java: java.util.PriorityQueue

Inverted indexes

Inverted index in computer science (also referred to as postings file or inverted file) is an index data structure storing a mapping from content, such as words or numbers, to its locations in a database file, or in a document or a set of documents (named in contrast to a forward index, which maps from documents to content). The purpose of an inverted index is to allow fast full text searches, at a cost of increased processing when a document is added to the database. The inverted file may be the database file itself, rather than its index. It is the most popular data structure used in document retrieval systems, used on a large scale for example in search engines. Additionally, several significant general-purpose mainframe-based database management systems have used inverted list architectures

The Fourier transform
  • betterexplained.com/articles/an-interactive-guide-to-the-fourier-transform/

  • usecase: Shazam

Parallel algorithms

a parallel algorithm, as opposed to a traditional serial algorithm, is an algorithm which can be executed a piece at a time on many different processing devices, and then combined together again at the end to get the correct result. Many parallel algorithms are executed concurrently – though in general concurrent algorithms are a distinct concept – and thus these concepts are often conflated, with which aspect of an algorithm is parallel and which is concurrent not being clearly distinguished. Further, non-parallel, non-concurrent algorithms are often referred to as "sequential algorithms", by contrast with concurrent algorithms.

MapReduce

MapReduce is a programming model and an associated implementation for processing and generating big data sets with a parallel, distributed algorithm on a cluster.

A MapReduce program is composed of a map procedure (or method), which performs filtering and sorting (such as sorting students by first name into queues, one queue for each name), and a reduce method, which performs a summary operation (such as counting the number of students in each queue, yielding name frequencies). The "MapReduce System" (also called "infrastructure" or "framework") orchestrates the processing by marshalling the distributed servers, running the various tasks in parallel, managing all communications and data transfers between the various parts of the system, and providing for redundancy and fault tolerance.

Probabilistic data structures

Probabilistic data structures can’t give you a definite answer, instead they provide you with a reasonable approximation of the answer and a way to approximate this estimation. They are extremely useful for big data and streaming application because they allow to dramatically decrease the amount of memory needed (in comparison to data structures that give you exact answers).

In majority of the cases these data structures use hash functions to randomize the items. Because they ignore collisions they keep the size constant, but this is also a reason why they can’t give you exact values. The advantages they bring: - they use small amount of memory (you can control how much) - they can be easily parallelizable (hashes are independent) - they have constant query time (not even amortized constant like in dictionary)

Also see: highlyscalable.wordpress.com/2012/05/01/probabilistic-structures-web-analytics-data-mining

Bloom filters

A Bloom filter is a space-efficient probabilistic data structure, that is used to test whether an element is a member of a set. False positive matches are possible, but false negatives are not – in other words, a query returns either "possibly in set" or "definitely not in set". Elements can be added to the set, but not removed (though this can be addressed with a "counting" filter); the more elements that are added to the set, the larger the probability of false positives. Bloom’s technique uses a smaller hash area but still eliminates most unnecessary accesses. For example, a hash area only 15% of the size needed by an ideal error-free hash still eliminates 85% of the disk accesses.

Count–min sketch

In computing, the count–min sketch (CM sketch) is a probabilistic data structure that serves as a frequency table of events in a stream of data. It uses hash functions to map events to frequencies, but unlike a hash table uses only sub-linear space, at the expense of overcounting some events due to collisions.

Count–min sketches are essentially the same data structure as the counting Bloom filters introduced in 1998 by Fan et al. However, they are used differently and therefore sized differently: a count-min sketch typically has a sublinear number of cells, related to the desired approximation quality of the sketch, while a counting Bloom filter is more typically sized to match the number of elements in the set.

Also see: stackoverflow.com/q/6811351

HyperLogLog

HyperLogLog is an algorithm for the count-distinct problem, approximating the number of distinct elements in a multiset. Calculating the exact cardinality of a multiset requires an amount of memory proportional to the cardinality, which is impractical for very large data sets. Probabilistic cardinality estimators, such as the HyperLogLog algorithm, use significantly less memory than this, at the cost of obtaining only an approximation of the cardinality. The HyperLogLog algorithm is able to estimate cardinalities of > 10^9 with a typical accuracy of 2%, using 1.5 kB of memory. Also a probabilistic data structure.

Cryptographic hash functions

A cryptographic hash function is a special class of hash function that has certain properties which make it suitable for use in cryptography. It is a mathematical algorithm that maps data of arbitrary size to a bit string of a fixed size (a hash) and is designed to be a one-way function, that is, a function which is infeasible to invert. The only way to recreate the input data from an ideal cryptographic hash function’s output is to attempt a brute-force search of possible inputs to see if they produce a match, or use a rainbow table of matched hashes. Bruce Schneier has called one-way hash functions "the workhorses of modern cryptography". The input data is often called the message, and the output (the hash value or hash) is often called the message digest or simply the digest.

The ideal cryptographic hash function has five main properties: - it is deterministic so the same message always results in the same hash - it is quick to compute the hash value for any given message - it is infeasible to generate a message from its hash value except by trying all possible messages - a small change to a message should change the hash value so extensively that the new hash value appears uncorrelated with the old hash value - it is infeasible to find two different messages with the same hash value

Diffie-Hellman key exchange

Diffie–Hellman key exchange (DH) is a method of securely exchanging cryptographic keys over a public channel. Diffie–Hellman is used to secure a variety of Internet services. However, research published in October 2015 suggests that the parameters in use for many DH Internet applications at that time are not strong enough to prevent compromise by very well-funded attackers, such as the security services of large governments.

Simplex algorithm and Linear programming

In mathematical optimization, Dantzig’s simplex algorithm (or simplex method) is a popular algorithm for linear programming.

All the graph algorithms can be done through linear programming instead. Linear programming is a much more general framework, and graph problems are a subset of that.

Web Tech

Javascript

Basics
On page script
<script type="text/javascript">  ...
</script>
Include external JS file
<script src="filename.js"></script>
Functions
function addNumbers(a, b) {
    return a + b; ;
}

x = addNumbers(1, 2);
Edit DOM element
document.getElementById("elementID").innerHTML = "Hello World!";
Output
console.log(a);             // write to the browser console
document.write(a);          // write to the HTML
alert(a);                   // output in an alert box
confirm("Really?");         // yes/no dialog, returns true/false depending on user click
prompt("Your age?","0");    // input dialog. Second argument is the initial value
Comments
/* Multi line
   comment */
// One line
Variables
var a;                          // variable
var b = "init";                 // string
var c = "Hi" + " " + "Joe";     // = "Hi Joe"
var d = 1 + 2 + "3";            // = "33"
var e = [2,3,5,8];              // array
var f = false;                  // boolean
var g = /()/;                   // RegEx
var h = function(){};           // function object
const PI = 3.14;                // constant
var a = 1, b = 2, c = a + b;    // one line
let z = 'zzz';                  // block scope local variable
Strict mode
"use strict";   // Use strict mode to write secure code
x = 1;          // Throws an error because variable is not declared
Values
false, true                     // boolean
18, 3.14, 0b10011, 0xF6, NaN    // number
"flower", 'John'                // string
undefined, null , Infinity      // special
Operators
a = b + c - d;      // addition, substraction
a = b * (c / d);    // multiplication, division
x = 100 % 48;       // modulo. 100 / 48 remainder = 4
a++; b--;           // postfix increment and decrement
Bitwise operators
& 	  AND  	                   5 & 1 (0101 & 0001)	1 (1)
| 	  OR  	                   5 | 1 (0101 | 0001)	5 (101)
~ 	  NOT  	                   ~ 5 (~0101)	10 (1010)
^ 	  XOR  	                   5 ^ 1 (0101 ^ 0001)	4 (100)
<< 	  left shift  	           5 << 1 (0101 << 1)	10 (1010)
>> 	  right shift  	           5 >> 1 (0101 >> 1)	2 (10)
>>> 	zero fill right shift  	 5 >>> 1 (0101 >>> 1)	2 (10)
Arithmetic
a * (b + c)         // grouping
person.age          // member
person[age]         // member
!(a == b)           // logical not
a != b              // not equal
typeof a            // type (number, object, function...)
x << 2  x >> 3      // minary shifting
a = b               // assignment
a == b              // equals
a != b              // unequal
a === b             // strict equal
a !== b             // strict unequal
a < b   a > b       // less and greater than
a <= b  a >= b      // less or equal, greater or eq
a += b              // a = a + b (works with - * %...)
a && b              // logical and
a || b              // logical or
Strings
var abc = "abcdefghijklmnopqrstuvwxyz";
var esc = 'I don\'t \n know';   // \n new line
var len = abc.length;           // string length
abc.indexOf("lmno");            // find substring, -1 if doesn't contain
abc.lastIndexOf("lmno");        // last occurance
abc.slice(3, 6);                // cuts out "def", negative values count from behind
abc.replace("abc","123");       // find and replace, takes regular expressions
abc.toUpperCase();              // convert to upper case
abc.toLowerCase();              // convert to lower case
abc.concat(" ", str2);          // abc + " " + str2
abc.charAt(2);                  // character at index: "c"
abc[2];                         // unsafe, abc[2] = "C" doesn't work
abc.charCodeAt(2);              // character code at index: "c" -> 99
abc.split(",");                 // splitting a string on commas gives an array
abc.split("");                  // splitting on characters
128.toString(16);               // number to hex(16), octal (8) or binary (2)
Numbers and Math
var pi = 3.141;
pi.toFixed(0);          // returns 3
pi.toFixed(2);          // returns 3.14 - for working with money
pi.toPrecision(2)       // returns 3.1
pi.valueOf();           // returns number
Number(true);           // converts to number
Number(new Date())      // number of milliseconds since 1970
parseInt("3 months");   // returns the first number: 3
parseFloat("3.5 days"); // returns 3.5
Number.MAX_VALUE        // largest possible JS number
Number.MIN_VALUE        // smallest possible JS number
Number.NEGATIVE_INFINITY// -Infinity
Number.POSITIVE_INFINITY// Infinity
Data Types
var age = 18;                           // number
var name = "Jane";                      // string
var name = {first:"Jane", last:"Doe"};  // object
var truth = false;                      // boolean
var sheets = ["HTML","CSS","JS"];       // array
var a; typeof a;                        // undefined
var a = null;                           // value null
Objects
var student = {                 // object name
    firstName:"Jane",           // list of properties and values
    lastName:"Doe",
    age:18,
    height:170,
    fullName : function() {     // object function
       return this.firstName + " " + this.lastName;
    }
};

student.age = 19;           // setting value
student[age]++;             // incrementing
name = student.fullName();  // call object function
Arrays
var dogs = ["Bulldog", "Beagle", "Labrador"];
var dogs = new Array("Bulldog", "Beagle", "Labrador");  // declaration

alert(dogs[1]);             // access value at index, first item being [0]
dogs[0] = "Bull Terier";    // change the first item

for (var i = 0; i < dogs.length; i++) {     // parsing with array.length
    console.log(dogs[i]);
}
Methods
dogs.toString();                        // convert to string: results "Bulldog,Beagle,Labrador"
dogs.join(" * ");                       // join: "Bulldog * Beagle * Labrador"
dogs.pop();                             // remove last element
dogs.push("Chihuahua");                 // add new element to the end
dogs[dogs.length] = "Chihuahua";        // the same as push
dogs.shift();                           // remove first element
dogs.unshift("Chihuahua");              // add new element to the beginning
delete dogs[0];                         // change element to undefined (not recommended)
dogs.splice(2, 0, "Pug", "Boxer");      // add elements (where, how many to remove, element list)
var animals = dogs.concat(cats,birds);  // join two arrays (dogs followed by cats and birds)
dogs.slice(1,4);                        // elements from [1] to [4-1]
dogs.sort();                            // sort string alphabetically
dogs.reverse();                         // sort string in descending order
x.sort(function(a, b){return a - b});   // numeric sort
x.sort(function(a, b){return b - a});   // numeric descending sort
highest = x[0];                         // first item in sorted array is the lowest (or highest) value
x.sort(function(a, b){return 0.5 - Math.random()});     // random order sort

concat, copyWithin, every, fill, filter, find, findIndex, forEach, indexOf, isArray, join, lastIndexOf, map, pop, push, reduce, reduceRight, reverse, shift, slice, some, sort, splice, toString, unshift, valueOf
If - Else
if ((age >= 14) && (age < 19)) {        // logical condition
    status = "Eligible.";               // executed if condition is true
} else {                                // else block is optional
    status = "Not eligible.";           // executed if condition is false
}
Switch Statement
switch (new Date().getDay()) {      // input is current day
    case 6:                         // if (day == 6)
        text = "Saturday";
        break;
    case 0:                         // if (day == 0)
        text = "Sunday";
        break;
    default:                        // else...
        text = "Whatever";
}
Loops
For Loop
for (var i = 0; i < 10; i++) {
    document.write(i + ": " + i*3 + "<br />");
}

var sum = 0;
for (var i = 0; i < a.length; i++) {
    sum + = a[i];
}               // parsing an array

html = "";
for (var i of custOrder) {
    html += "<li>" + i + "</li>";
}
While Loop
var i = 1;                      // initialize
while (i < 100) {               // enters the cycle if statement is true
    i *= 2;                     // increment to avoid infinite loop
    document.write(i + ", ");   // output
}
Do While Loop
var i = 1;                      // initialize
do {                            // enters cycle at least once
    i *= 2;                     // increment to avoid infinite loop
    document.write(i + ", ");   // output
} while (i < 100)               // repeats cycle if statement is true at the end
Break
for (var i = 0; i < 10; i++) {
    if (i == 5) { break; }          // stops and exits the cycle
    document.write(i + ", ");       // last output number is 4
}
Continue
for (var i = 0; i < 10; i++) {
    if (i == 5) { continue; }       // skips the rest of the cycle
    document.write(i + ", ");       // skips 5
}
Errors
try {                           // block of code to try
    undefinedFunction();
}
catch(err) {                    // block to handle errors
    console.log(err.message);
}
Throw error
throw "My error message";    // throw a text
Input validation
var x = document.getElementById("mynum").value; // get input value
try {
    if(x == "")  throw "empty";                 // error cases
    if(isNaN(x)) throw "not a number";
    x = Number(x);
    if(x > 10)   throw "too high";
}
catch(err) {                                    // if there's an error
    document.write("Input is " + err);          // output error
    console.error(err);                         // write the error in console
}
finally {
    document.write("</br />Done");              // executed regardless of the try / catch result
}

Error name values
RangeError A number is "out of range"
ReferenceError An illegal reference has occurred
SyntaxError A syntax error has occurred
TypeError A type error has occurred
URIError An encodeURI() error has occurred
Global functions
eval();                     // executes a string as if it was script code
String(23);                 // return string from number
(23).toString();            // return string from number
Number("23");               // return number from string
decodeURI(enc);             // decode URI. Result: "my page.asp"
encodeURI(uri);             // encode URI. Result: "my%page.asp"
decodeURIComponent(enc);    // decode a URI component
encodeURIComponent(uri);    // encode a URI component
isFinite();                 // is variable a finite, legal number
isNaN();                    // is variable an illegal number
parseFloat();               // returns floating point number of string
parseInt();                 // parses a string and returns an integer
Promises
function sum (a, b) {
   return Promise(function (resolve, reject) {
     setTimeout(function () {                                       // send the response after 1 second
       if (typeof a !== "number" || typeof b !== "number") {        // testing input types
         return reject(new TypeError("Inputs must be numbers"));
       }
       resolve(a + b);
     }, 1000);
   });
}
var myPromise = sum(10, 5);
myPromsise.then(function (result) {
  document.write(" 10 + 5: ", result);
  return sum(null, "foo");              // Invalid data and return another promise
}).then(function () {                   // Won't be called because of the error
}).catch(function (err) {               // The catch handler is called instead, after another second
  console.error(err);                   // => Please provide two numbers to sum.
});
States
  • pending, fulfilled, rejected

Properties
  • Promise.length, Promise.prototype

Methods
  • Promise.all(iterable), Promise.race(iterable), Promise.reject(reason), Promise.resolve(value)

JSON
var str = '{"names":[' +                    // crate JSON object
'{"first":"Hakuna","lastN":"Matata" },' +
'{"first":"Jane","lastN":"Doe" },' +
'{"first":"Air","last":"Jordan" }]}';
obj = JSON.parse(str);                      // parse
document.write(obj.names[1].first);         // access
Send
var myObj = { "name":"Jane", "age":18, "city":"Chicago" };  // create object
var myJSON = JSON.stringify(myObj);                         // stringify
window.location = "demo.php?x=" + myJSON;                   // send to php
Storing and retrieving
myObj = { "name":"Jane", "age":18, "city":"Chicago" };
myJSON = JSON.stringify(myObj);                 // storing data
localStorage.setItem("testJSON", myJSON);

text = localStorage.getItem("testJSON");        // retrieving data
obj = JSON.parse(text);
document.write(obj.name);
Dates
Thu Nov 01 2018 17:49:50 GMT+0100 (Central European Standard Time)

var d = new Date();

1541090990284 miliseconds passed since 1970

Number(d)

Date("2017-06-23");                 // date declaration
Date("2017");                       // is set to Jan 01
Date("2017-06-23T12:00:00-09:45");  // date - time YYYY-MM-DDTHH:MM:SSZ
Date("June 23 2017");               // long date format
Date("Jun 23 2017 07:45:00 GMT+0100 (Tokyo Time)"); // time zone
Get Times
var d = new Date();
a = d.getDay();     // getting the weekday

getDate();          // day as a number (1-31)
getDay();           // weekday as a number (0-6)
getFullYear();      // four digit year (yyyy)
getHours();         // hour (0-23)
getMilliseconds();  // milliseconds (0-999)
getMinutes();       // minutes (0-59)
getMonth();         // month (0-11)
getSeconds();       // seconds (0-59)
getTime();          // milliseconds since 1970
Setting part of a date
var d = new Date();
d.setDate(d.getDate() + 7); // adds a week to a date

setDate();          // day as a number (1-31)
setFullYear();      // year (optionally month and day)
setHours();         // hour (0-23)
setMilliseconds();  // milliseconds (0-999)
setMinutes();       // minutes (0-59)
setMonth();         // month (0-11)
setSeconds();       // seconds (0-59)
setTime();          // milliseconds since 1970)
Regular Expressions
var a = str.search(/CheatSheet/i);
Modifiers
i perform case-insensitive matching
g perform a global match
m perform multiline matching
Patterns
\ Escape character
\d find a digit
\s find a whitespace character
\b find match at beginning or end of a word
n+ contains at least one n
n* contains zero or more occurrences of n
n? contains zero or one occurrences of n
^ Start of string
$ End of string
\uxxxx find the Unicode character
. Any single character
(a|b) a or b
(...) Group section
[abc] In range (a, b or c)
[0-9] any of the digits between the brackets
[^abc] Not in range
\s White space
a? Zero or one of a
a* Zero or more of a
a*? Zero or more, ungreedy
a+ One or more of a
a+? One or more, ungreedy
a{2} Exactly 2 of a
a{2,} 2 or more of a
a{,5} Up to 5 of a
a{2,5} 2 to 5 of a
a{2,5}? 2 to 5 of a, ungreedy
[:punct:] Any punctu­ation symbol
[:space:] Any space character
[:blank:] Space or tab
Events
<button onclick="myFunction();">
   Click here
</button>
Mouse

onclick, oncontextmenu, ondblclick, onmousedown, onmouseenter, onmouseleave, onmousemove, onmouseover, onmouseout, onmouseup

Keyboard
onkeydown, onkeypress, onkeyup
Frame
onabort, onbeforeunload, onerror, onhashchange, onload, onpageshow, onpagehide, onresize, onscroll, onunload
Form
onblur, onchange, onfocus, onfocusin, onfocusout, oninput, oninvalid, onreset, onsearch, onselect, onsubmit
Drag
ondrag, ondragend, ondragenter, ondragleave, ondragover, ondragstart, ondrop
Clipboard
oncopy, oncut, onpaste
Media
onabort, oncanplay, oncanplaythrough, ondurationchange, onended, onerror, onloadeddata, onloadedmetadata, onloadstart, onpause, onplay, onplaying, onprogress, onratechange, onseeked, onseeking, onstalled, onsuspend, ontimeupdate, onvolumechange, onwaiting
Animation
animationend, animationiteration, animationstart
Miscellaneous
transitionend, onmessage, onmousewheel, ononline, onoffline, onpopstate, onshow, onstorage, ontoggle, onwheel, ontouchcancel, ontouchend, ontouchmove, ontouchstart

jquery

Basics

Include

<script src="https://code.jquery.com/jquery-latest.min.js" type="text/javascript"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.0/jquery.min.js"></script>

Syntax

$(document).ready(function(){
    $(".demo").click(function(){
        $(this).hide(200);
    });
});

$(function(){
   // Short Document Ready
});

Each

$(".demo").each(function() {                // parse each .demo element
    document.write($(this).text() + "\n");  // output their text
});

Trigger

$("a#mylink").trigger("click"); // triggers event on an element

noConflict

var jq = $.noConflict();            // avoid conflict with other frameworks also using the dollar sign
jq(document).ready(function(){
    jq("#demo").text("Hello World!");
});
Traversing
$("#demo").parent();                // accessing direct parent
$("span").parent().hide();          // changing parent color
$("#demo").parents();               // all ancestors of the element
$("#demo").parentsUntil("#demo2");  // all ancestors between two - demo is inside demo2
$("#demo").children();              // all direct children
$("#demo").children(".first");      // all direct children having a specified class
$("#demo").find("span");            // all span elements inside #demo
$("#demo").find("*");               // all descendants
$("#demo").siblings("span");        // span siblings of #demo
$("#demo").next();                  // the next sibling
$("p").nextAll();                   // all next siblings
$("#demo").nextUntil("#demo2");     // siblings between two arguments
$("#demo").prev();                  // the previous sibling
$("p").prevAll();                   // all previous siblings
$("#demo").prevUntil("#demo2");     // previous siblings between two arguments

Filtering

$("span strong").first();   // first strong in first span
$("span strong").last();    // last strong in last span
$("div").eq(9);             // element with a specific index
$("div").filter(".big");    // all div elements with .big class
$("div").not(".big");       // opposite of filter
Selectors
$("*")                      // all elements
$("p.demo")                 // <p> elements with class="intro"
$("p:first")                // the first <p> element
$("p span")                 // span, descendant of p
$("p > span")               // span, direct child of p
$("p + span")               // span immediately proceeded by a p
$("p ~ span")               // strong element proceeded by p
$("ul li:first")            // the first <li> element of the first <ul>
$("ul li:first-child")      // the first <li> element of every <ul>
$("ul li:nth-child(3)")     // third child
$("[href]")                 // any element with an href attribute
$("a[target='_blank']")     // <a> elements with a target "_blank" attribute
$("a[target!='_blank']")    // <a> elements with a target attribute value other than "_blank"
$(":input")                 // all form elements
$(":button")                // <button> and <input> elements of type="button"
$("tr:even")                // even <tr> elements
$("tr:odd")                 // odd <tr> elements
$("span:parent")            // element which has child element
$("span:contains('demo')")  // element conaining the specified text

Actions

$(selector).action()
$(this).hide()      // the current element
$("div").hide()     // all <div> elements
$(".demo").hide()   // all elements with class="demo"
$("#demo").hide()   // the element with id="demo"
Events
$(".demo").click(function(){
    $(this).hide(200);
});

.bind()

$(document).ready(function() {              // attach a handler to an event for the elements
    $("#demo").bind('blur', function(e) {
        //dom event fired
    });
});
  • Mouse: scroll, click, dblclick, mousedown, mouseup, mousemove, mouseover, mouseout, mouseenter, mouseleave, load, resize, scroll, unload, error

  • Keyboard: keydown, keypress, keyup

  • Form: submit, change, focus, blur

  • DOM Element: blur, focus, focusin, focusout, change, select, submit

  • Browser: load, resize, scroll, unload, error

DOM Manipulation

Content

$("#demo").text();                  // returns text content
$("#demo").html();                  // returns content, including HTML markup
$("#demo").val();                   // returns field value
$("#demo").html('Hey <em>yo</em>'); // sets HTML content

Attributes

$("#link").attr("href");                    // get an attribute
$("#link").attr("href",'https://htmlg.com'); // set attribute
$("#link").attr({
    "href" : "https://htmlg.com",            // setting multiple attributes
    "title" : "HTML Editor"
});
$("#link").attr("href", function(i, origValue){
    return origValue + "/help";             // callback function gets and changes the attribute
});

Add

$(".demo").prepend("Yo!");          // adds content at the beginning in the selected elements
$(".demo").append("<em>Hey!</em>"); // adds content at the end in the selected elements
$(".demo").before("Cheers");        // adds content before the selected elements
$(".demo").after("<em>Peace</em>"); // adds content after the selected elements

Remove

$("#demo").remove();            // removes the selected element
$("#demo").empty();             // removes children
$("div").remove(".cl1, .cl2");  // removes divs with the listed classes

Classes

$("#demo").addClass("big red"); // add class
$("h1, p").removeClass("red");  // remove class
$("#demo").toggleClass("big");  // toggle between adding and removing

CSS

$("#demo").css("background-color");     // returns CSS value
$("#demo").css("color", "blue");        // sets CSS rule
$("#demo").css({"color": "blue", "font-size": "20px"}); // sets multiple CSS properties

Dimensions - width, height, innerWidth, innerHeight, outerWidth, outerHeight - inner - includes padding - outer - includes padding and border

Ajax
$("#demo").load("file.txt h1.main");   // returns the h1 tag in the text file
$("#demo").load("file.txt", function(responseTxt, statusTxt, xhr){  // callback function
    if(statusTxt == "success") {
        document.write("Content loaded successfully!");
    } else {
        document.write("Error: " + xhr.status + ": " + xhr.statusText);
    }
});

$.get()

$.get("demo.asp", function(data, status){       //$.get(URL,callback);
    document.write("Data: " + data + "\nStatus: " + status);
});

$.post()

$.post("demo.asp",      // send HTTP POST request to a page and get the answer
{
    name: "John",       // send data
    age: 30
},
function(data, status){ //retreive response
    console.log("Data: " + data + "\nStatus: " + status);
});
Effects

Hide / Show

$("#demo").hide();      // sets to display: none
$("#demo").show(200);   // shows hidden elemnt with animation (speed)
$("#demo").toggle();    // toggle between show and hide

$( "#element" ).hide( "slow", function() {  // hide with callback function
    console.log( "Animation complete." );
});

Fade - fadeIn, fadeOut, fadeToggle, fadeTo

$("#demo").fadeIn();                // fade in a hidden element
$("#demo").fadeOut(300);            // fade out
$("#demo").fadeToggle("slow");      // toggle between fadeIn and hadeOut
$("#demo").fadeTo("slow", 0.25);    // fades to 0.25 opacity

Slide - slideDown, slideUp, slideToggle

$("#demo").slideDown();
$("#demo").slideUp("slow");
$("#demo").slideToggle();

Animate

$(selector).animate({params},speed,callback);

$("div").animate({
    opacity: '0.5',
    left: '200px',
    height: '200px'
});

stop() Method

$("#demo").stop();

$('#demo').mouseleave(function(event) {     // hover end
    $('.tab').stop().animate({              // stop() method
        opacity : '0.5',
        marginTop: '10px'
    }, 500, function() {                    // animation complete
        $('#demo').removeClass('hovered');  // callback function
    });
});
$('#demo').mouseover(function(event) {      // hover begin
    $('.tab').stop().animate({              // stop() method
        opacity : '1',
        marginTop: '0px'
    }, 300, function() {                    // animation complete
        $('#demo').addClass('hovered');     // callback function
    });
});

Chaining

$("#demo").css("backgroundColor", "green").slideUp(500).slideDown(500);

Books

The Clean Coder

ISBN: 978-0137081073
Tags: Software
Date read:
Rating:

Description: Great software isn’t written by machines. It is written by professionals with an unshakable commitment to craftsmanship. The Clean Coder will help you become one of them-and earn the pride and fulfillment that they alone possess.

Example 1. ▶ Notes & Quotes
Chapter 1: Professionalism
  • Do no harm to function or structure of the code: don’t introduce bugs, make sure you code is tested. If it can’t be tested, write it in a way that is testable. If testing is too cumbersome, automate the testing.

  • A true professional knows that delivering function at the expense of structure is a fool’s errand.

  • Every time you look at a module you should look for ways to make small, lightweight changes to make it better. AKA, leave code better than you found it even if you didn’t write it.

  • Your career is YOUR responsibility, not your employer’s. You owe your employer 40 hours a week, make sure you put in an extra some hours for yourself (reading, practicing, learning) to hone your skills and make sure you are developing in the areas you want to be developing in.

  • Know your field and know it well. Know design patterns and principles, methods, practices.

  • Practice. Practice. Practice. True professionals keep their skills sharp and ready. Musicians don’t get better by performing (doing your job), they get better by practicing (outside of work). That same rule applies to engineers.

Minimal list of the things that every software professional should be conversant with:
  • Design patterns. You ought to be able to describe all 24 patterns in the GOF book and have a working knowledge of many of the patterns in the POSA books.

  • Design principles. You should know the SOLID principles and have a good understanding of the component principles.

  • Methods. You should understand XP, Scrum, Lean, Kanban, Waterfall, Structured Analysis, and Structured Design.

  • Disciplines. You should practice TDD, Object-Oriented design, Structured Programming, Continuous Integration, and Pair Programming.

  • Artifacts: You should know how to use: UML, DFDs, Structure Charts, Petri Nets, State Transition Diagrams and Tables, flow charts, and decision tables.

When starting a project in a new domain, read a book or two on the topic. Interview your customer and users about the foundation and basics of the domain. Spend some time with the experts, and try to understand their principles and values. It is the worst kind of unprofessional behavior to simply code from a spec without understanding why that spec makes sense to the business. Rather, you should know enough about the domain to be able to recognize and challenge specification errors.

Chapter 2: Saying NO
  • True professionals have the courage to say no to their managers. Pursue and defend your objectives as aggressively as you can. If you know full well that getting the job done by X date is not doable, but you still say “I’ll try” then you are not doing your job right.

  • Your manager is counting on you to defend your objectives and not just agree with everything that she says. Be assertive. Both you and your manager need to get to the best possible outcome through negotiation.

  • Conversations might be adversarial and uncomfortable — but that’s all in the pursuit of a mutually agreeable solution and the best possible outcome (the ultimate goal).

  • When it comes not being able to meet the deadline, the WHY is less important than the FACT that you can’t meet it.

  • Make sure you have documentation (memos) for high stake deliverables/situations (CYA)

Chapter 3: Saying YES
  • Three parts to making a commitment: Say you’ll do it, mean it, actually do it.

  • Recognize lack of commitment phrases and words in others and yourself. They include: “need/should”, “hope/wish”, “Let’s meet sometime/Let’s finish this thing.”.

  • The secret ingredient to recognizing what a sincere commitment sounds like is to look for phrases that resemble “I will do something… by this certain date…”

  • If you rely on someone else to get your job done, do what you can to get what you need to move forward. Don’t let them be a blocker.

  • Bring up blockers or red flags as soon as they come up — communicate. If you don’t tell anyone about the potential problem as soon as possible, you’re not giving anyone a chance to help you follow through on your commitment.

  • Professionals are not required to say “yes” to everything that is asked of them. However, they should work hard to find creative ways to make the “yes” possible (e.g., reducing scope)

Chapter 4: Coding
  • Coding "requires a level of concentration and focus that few other disciplines require." A clean coder codes only if s/he can guarantee enough focus. Distractions (personal, environmental, or whatever) are a problem. Over-time is a problem.

  • Flow ("the Zone") is not as good as people think: You will be locally productive, but will often lose sight of the bigger picture and possibly produce not-so-good designs.

  • Music may be a distraction (even if you don’t think so).

  • Interruptions are bad distractions. Pair programming is helpful to cope with them. TDD helps to make the pre-interruption context reproducible.

  • If you have writer’s block, start pair programming.

  • Make sure you take in enough creative input, e.g. reading fiction books.

  • You have to find ways to minimize the time spent debugging - TDD.

  • Coding is a marathon, not a sprint, so conserve your energy and creativity. Go home when it’s time, even in the middle of some-thing important. Showers and cars are prob-lem-solving resources, too!

  • Continuously re-estimate your best/likely/worst completion time and speak up as soon as you recognize you will likely be late. Do not allow anyone to rush you (see above). Consider overtime only for a short stretch (2 weeks max.) and only if there is a fallback plan as well. Use a proper definition of "done", with sufficiently high quality requirements.

The Passionate Programmer

ISBN: 978-1934356340
Tags: Software, Self-help, Buisness
Date read: 17.5.2019.
Rating: 3/5

Description: Success in today’s IT environment requires you to view your career as a business endeavor. In this book, you’ll learn how to become an entrepreneur, driving your career in the direction of your choosing. You’ll learn how to build your software development career step by step, following the same path that you would follow if you were building, marketing, and selling a product. After all, your skills themselves are a product.

Example 2. ▶ Notes & Quotes
Intro:
  • Chose your market: Pick the tech and business domains you focus on consistently and deliberatly

  • Invest in your product(tools): expand your knowledge and skills

  • Execute: utilize your skills to deliver

  • Market: build a personal brand

Make a list of early, middle, and late adoption technologies based on today’s market. Map them out on paper from left to right; the left is bleeding edge, and the right is filled by technologies that are in their sunsets. Push yourself to find as many technologies in each part of the spectrum as possible. Be as granular as possible about where in the curve they fall in relation to one another. When you have as many technologies mapped out as you can think of, mark the ones that you consider yourself strong in. Then, perhaps in a different color, mark the ones that you have some experience with but aren’t authoritative on. Where are most of your marks on the adoption curve? Do they clump? Are they spread evenly across? Are there any technologies around the far edges that you have some special interest in?

Exploit market imbalances.

Aquire trade magazines, websites etc for your company industry - watch out for big news. (industry != software but rather industry youre writting software for, ie. cattle)

Make mindmaps - one draft and then one finished

Talk to the duck.

We build to learn, not learn to build.

Find a “be worst” situation for yourself - meetups, workshops, open source projects etc.

Generalists are rare and therefore precious.

Do same thing in two different ways, for example small project in both list and python, or c# and java, or with and without stl.

Keep log - for eg. not in the morning on 1-10 scale how you feel and plan how to make it 10 next morning.

Find what you dont know and learn. Disect things.

Be a mentor.

Practice your limits - competitive programming, topcoder, kata’s etc

PM methodologies: PMBoK, SixSigma

Mine existing code for insights

Code reviews every week - your company’s code, open source code segments

If you treat your projects as a race you’ll finish them fast (thin weekend hackaton). Parkinson’s law. Turn multimonth project into a week task.

The mindreading trick if done well leads to people depending on you. If you do what your customers ask for, they’ll be happy. If you mae things before they even ask for them, you will delight them. however thread carefully, you can fuck up easily.

Have an accomplishment to report every day. This becomes a habit, rather than a major production.

Be ambitious but dont wear it on your sleeve.

Implement self checking systems. ie make team compete between themself but as all vs 1, for every teammate. Constructive spiral.

beware of being blinded by your own success.

Keep system components standalone to be able to swap servers, users, team…​ as you wish.

Eight hour burn: work so hard you cant continue after 8 hours.

Budget yout work hours - work less to do more. Work intensivly, then take a lunch or nap or exercise, then work some more.

Defensive planing and programming. (stability and security in the first plan, then rest)

Saying "yes" is addictive and destructive habit.

Note down your commitments list and examinate it daily.

Never panic. No matter how bad it is try to keep clear head. When you start panicing think of that old computer illiterate lady. Analyse situation from third persons perspective, just as you were helping someone. Log your panics and progress on them.

Write status reports daily, weekly, monthly, yearly.

Take different groups and list them. Next to each, write down which of your attributes is most likely to drive that group’s perception of you. Here’s an example:

groups

Be understanding of people who dont know or understand yet. Dont make them fear you.

You are what you can explain. Learn how to write and how to talk. (touch typing, logbook, diary)

Face to face communication is important. Have an office, buisness is personal. Second best is calls, dont email what you can call.

Businesses are interested in business results. Show what you did with focus on why it matters (for business).

Write a textfile full of possible weblog topics. These are your future articles that you are going to write. Shoot for ideas that you can write about in 10-20min. In articles link to relevant stuff. Write daily for 3 weeks. Now cherry pick and tidy and publish best on usermoderated pages such as reddit or digg.

Build your brand. Make google searches say good things about you. Dont be a jerk online or offline.

Commit to open source. "Im contributor to python software" is so much better than listing python as a skill on CV.

Think of someone local you adore, try to find a way to start conversation (their speech, meetup, …​), especially if it makes you uncomfortable.

Your shiny new skills are already obsolete. Make room for at least two hours each week in order to research new technologies and to start to develop skills in them. Do hands-on work with these new technologies. Build simple applications. Prototype new-tech versions of the hard bits of your current-tech projects to understand what the differences are and what the new technologies enable. Put this time on your schedule. Don’t let yourself miss it.

If you’re a programmer, try a day or two of doing your job as if you were a tester or a project manager. What are the many roles that you might play from day to day that you have never explicitly considered? Make a list, and try them on for size. Spend a day on each. You might not even change your actual work output, but you’ll see your work differently.

Focus on doing, not on being done. In his book The Miracle of Mindfulness Thich Naht Hanh presents a suggestion: the next time you have to wash the dishes, don’t wash them to get them done. Try to enjoy the experience of washing the dishes. Don’t focus on finishing them. Focus on the act of washing them itself. Doing the dishes is a mundane task that almost nobody savors. Software developers have a lot of similar drudgery to get through in the average day, such as time tracking and expense reporting, for example. The next time you have to do a task like this, see whether you can find a way to focus on the task as you do it instead of anxiously rushing to finish it.

A personal product road map not only can help you stay on track, constantly evolving, but it can also show you the bigger picture of what you have to offer. It can show you that no single feature stands alone. Each new investment is part of a larger whole. Some work fabulously well together. Others require too much of a mental leap for potential employers. Is he a system administrator or a graphic designer? Is she an application architect or a QA automation guru? Although it’s definitely OK to learn diverse skills — it expands your thinking — it’s also a good idea to think about the story your skillset tells. Without a road map, your story might look more like a Jack Kerouac novel than a cohesive set of logically related capabilities. Without a road map, you might even actually get lost. Adjust roadmap/cv/experiences you shard based on person you are conversing with.

If you sit and watch a flower bloom, it will take a long time to notice that anything has happened. However, if you leave and come back in two days, you’ll see something very noticeably different from when you left. You might look at yourself in the metaphorical mirror each day and not see an ounce of change. Keep a journal and ask peope around you for feedback.

Make a list of the difficult or complex improvements you’d like to make; they can be personal or professional. It’s OK if you have a fairly long list. Now, for each item in the list, think about what you could do today to make yourself or that item better than yesterday. Tomorrow, look at the list again. Was yesterday better than the day before? How can you make today better? Do it again the next day. Put it on your calendar. Spend two minutes thinking about this each morning.