Would you hire GPT-3 instead of a developer?

CoPilot, the AI pair programmer by OpenAI and GitHub/Microsoft, is generally available, I've been very slow in trying it out and so far have recommended our developers DON'T use it on our internal codebases due to IP and concerns (see end).

But, you can play with the same system, to test it, without giving it access to your IDE or internal codebases, just by signing up to OpenAI's Playground - where you get $18 of free credits to ask questions of this huge AI model.

So, I did. I fed it a few typical JavaScript interview questions and was impressed by the results:

Create a React component which lets you up and down vote, with buttons, and keeps a running total of the number of up and down votes.

class UpDownVotes extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
      upvotes: 0,
      downvotes: 0
    }
  }
  
  upVote = () => {
    this.setState({
      upvotes: this.state.upvotes + 1
    })
  }
  
  downVote = () => {
    this.setState({
      downvotes: this.state.downvotes + 1
    })
  }
  
  render() {
    return (
      <div>
        <button onClick={this.upVote}>Up Vote</button>
        <button onClick={this.downVote}>Down Vote</button>
        <p>Upvotes: {this.state.upvotes}</p>
        <p>Downvotes: {this.state.downvotes}</p>
      </div>
    )
  }
}

Not bad! Let's try something a little harder:

Create a javascript function SimplePassword(str) take the str parameter being passed and determine if it passes as a valid password that follows the list of constraints:
1. It must have a capital letter.
2. It must contain at least one number.
3. It must contain a punctuation mark or mathematical symbol.
4. It cannot have the word "password" in the string.
5. It must be longer than 7 characters and shorter than 31 characters.
If all the above constraints are met within the string, the your program should return the string true, otherwise your program should return the string false. For example: if str is "apple!M7" then your program should return "true".

function SimplePassword(str) { 
  if(str.length>7 && str.length<31){
    if(/^[0-9a-zA-Z!@#$%^&*()_+\-=\[\]{};':"\\|,.<>\/?]*$/.test(str)){
      var pass = str.split("");
      var hasNum = false;
      var hasLetter = false;
      var hasPunc = false;
      for(var i=0;i<pass.length;i++){
        if(/[0-9]/.test(pass[i]))
          hasNum=true;
        else if(/[a-zA-Z]/.test(pass[i]))
          hasLetter=true;
        else{
          hasPunc=true;
        }
      }
      if(hasNum&&hasLetter&&hasPunc){
        if(str.toLowerCase().search("password")==-1)
          return true;
        else
          return false;
      }
    }
  }
  return false;        
}

Yep, ok. What about a theory question?

What do you understand by Closures in JavaScript?

A closure is a function that makes use of variables defined 
in outer functions that have previously returned. 
In JavaScript, closures are created every time a function 
is created that uses variables from outside its scope.

Or a question about what NPM modules to use?

Which npm modules would you use to manage a database schema?

There are many npm modules that can be used to manage 
a database schema, but two of the most popular are 
sequelize and knex.

Ok, let's dig into that:

What are the pros and cons of sequelize vs knex?

There is no definitive answer to this question as it depends
on personal preferences and opinions. Some people may prefer
sequelize because it offers more features and is better suited
for larger projects, while others may prefer knex because it
is simpler and easier to use.

It's not just code, it can answer questions about git

What does the following command do?
git diff-tree –r {commit hash}

The command git diff-tree –r {commit hash} displays the
differences between the commit with the given hash and
its parents.

or deployment strategies

What deployment strategies do you know and have used before? How can a company deploy with no downtime?

There are a few deployment strategies:

1. Blue/Green Deployment: With this strategy, you have two identical
production environments. You deploy your updates to the environment 
that is not currently serving traffic (the "green" environment), and 
then once the updates are deployed and tested, you switch traffic over
to the green environment. This ensures that there is no downtime, as 
users will always be able to access the site/application.

2. Canary Deployment: This is similar to blue/green deployment, but 
instead of having two identical production environments, you have a 
production environment and a staging environment. You deploy your 
updates to the staging environment first, and then if everything looks 
good, you deploy the updates to production. This ensures that there is 
less risk, as you can test the updates in a non-production environment 
first.

3. A/B Testing: With this strategy, you deploy your updates to a small 
subset of users first, and then if everything looks good, you deploy the 
updates to all users. This helps to reduce the risk of deploying to all 
users at once, as you can test the updates with a smaller group first.

4. Rolling Deployment: This strategy involves slowly rolling out the 
updates to all users, rather than deploying all at once. This can help 
to reduce the risk of deploying to all users at once, as you can make 
sure that the updates are working as expected before deploying to all 
users.

It wasn't perfect, it missed parts of questions, it got confused when I used longer and more intricate problems - but over all, it would not fare badly against most entry level or even mid level candidates!

BUT. As mentioned, it's not all magic and rainbows - there are some serious questions about the ownership and licencing of the code it generates; like this one

and this one

and this one

Can AI learn from any public code online?

A question of fair use

The VergeDave Gershgorn

and this one

Is Github Copilot safe privacy wise? · Discussion #7163 · github-community/community

Hi I'm sure this is a stupid question and someone else probably asked it. But is Github Copilot safe to be used on private projects/repos. I'm sure you collect prediction analytics but that...

GitHubgithub-community

But...

Would you hire GPT-3 instead of a developer?

GPT-3 is not a developer, so it cannot be hired to do development work.

...but for how long?

And that crazy image at the top, that's another OpenAI (DALL-E) model, which instead of generating code/text, creates images. The image was inspired by the prompt "An AI doing a coding exam" and created using the NightCafe service. Its using DALL-E v1, and there is already a v2 out.