The big git and github Pull Request thread

[sofar]

In this thread you'll be able to ask, and find, all about working with git and github in the minetest community. Topics covered include anything related to using git for the purpose of developing minetest, mods, or any related project!

I'm splitting up the thread into several posts at the top that cover the important parts. Please follow the links here to jump to those topics directly:

1. Getting started with git and github

Why are we using github. How do I sign up, and what is it? What is this git program?

2. How to clone git repositories so you can use them

Includes references to run minetest from git, compiling it, using minetest_game, downloading mods from git, etc.

3. How to submit a PR to an existing git tree

Includes tips and hints on forking, pulling and pushing, submitting the PR's, and how to make working with PR's easy.

4. How to modify a git pull, commits

Covers topics needed to make working with Pull Requests fluid. Explains things like squashing and rebasing.

5. Maintaining a git project

Covers basics of setting up, merge and maintenance philosophies, best practices.

[sofar]

Why are we using github. How do I sign up, and what is it?

Git is a program that can store changes in files, so that you later can retrieve those changes. We use it because we often want to look back at those changes and understand how the changes work, why they were made, and who made them. It's not the only program that does this. Git comes from a long heritage of SCM programs that existed before, that all performed the same task.

The big difference with git (apart from being fully open source), is that git does an extremely good job at doing that in a collaborative and open setting, where everyone has a full copy of the code and can work independent, but can also exchange code between their own copies. It's therefore often called a Distributed Revision Control system.

github (https://github.com) is a company that provides hosting space for git repositories. It is a commercial company, but has traditionally flourished by hosting Open Source software repositories for free. While some people dislike the github interface, it provides a relatively simple way for developers to use the internals of the git protocol and methods to collaborate and comment on changes. This is done through the github web interface. In the github website, you can compare across forks and branches, and propose the changes in them to other branches (This is a Pull Request, or "PR") so that others may benefit from those changes as well.

How do I get git?

On Linux, please refer to your linux distro on how to install (packages, and) git. If you can find a distro that does not ship git, well, that's pretty rare :).

For Windows and OSX, you have several implementations, but the builds from https://git-scm.com/ are likely the best.

While I personally don't use them, some of the GUI clients listed on https://git-scm.com/ may be of interest. But I strongly suggest learning the basic git commands through the command line interface.

github signup

This should be fairly straigtforward, just visit https://github.com/ and register. Make sure to visit https://help.github.com/ and check out their long section of help pages that explain how to add keys, fork, etc.. Consider using SSH keys, and check this tread as well to get started: https://help.github.com/articles/generating-an-ssh-key/

[sofar]

Includes references to run minetest from git, compiling it, using minetest_game, downloading mods from git, etc.

If you want to change minetest, you should first switch to using a version of minetest from git, and not a package that is compiled by your distribution. For good measure, you'll always want to test your changes first, before making them public for others to see, so it's critical that you know how to get a copy of minetest or minetest_game from git, and test it.

For minetest, the easiest method is described in the README.txt of the minetest git tree. Please follow the instructions there.

If you follow the above instructions, you'll automatically also clone minetest_game. If you've skipped that step, and are looking to clone and modify minetest_game, instead do this:

For minetest_game, you'll want to create a clone of minetest_game in ~/.minetest/games/

Code: Select all

git clone git://github.com/minetest/minetest_game

(Please note that officially using minetest_game from git is only compatible with minetest from git. We never support using a minetest_game that's newer than the used minetest version).

Many mods can be cloned as well from github this way. Just

Code: Select all

cd ~/.minetest/mods

and

Code: Select all

git clone $url

to get started.

A small note about git URL's. The following are identical git URLs, and all point at the main minetest git repo:

Code: Select all

1. git://github.com/minetest/minetest
2. https://github.com/minetest/minetest
3. git@github.com:minetest/minetest

But, there are some major differences about these 3 urls:

• 1. git:// URL's are git's native protocol. It is highly efficient and designed for the fastest network cloning. But it can only be used to clone and pull, and you can't push changes this way to a remote repository.
• 2. https:// URL's use HTTP with SSL (encryption) to both pull and push (and clone). This method can anonymously checkout. However, HTTP is not a very efficient protocol, and if you authenticate, you'll have to type your password for every connection, so while this is secure, it's not fast, nor efficient to work with.
• 3. git@ URL's use SSH to clone, pull and push. It's very secure, and somewhat efficient because internally it uses the git protocol. There his however also no anonymous checking out. This is the preferred method for developers. It does require that you setup a github account and put SSH public keys in your github account, otherwise github doesn't recognize you.

[sofar]

Includes tips and hints on forking, pulling and pushing, submitting the PR's, and how to make working with PR's easy.

You've identified a place in minetest where you want to change something, and you want to submit these upstream so that everyone can enjoy them!

The following instructions should take you through the entire process:

Clone minetest and minetest_game

You should clone both projects as the two parts of minetest work together, and often changes in one require the other. The best way to test and verify that your changes work together is to fork and clone both and test them together.

Code: Select all

git clone git@github.com:mygithubname/minetest
git clone git@github.com:mygithubname/minetest_game

Compile and run your copy, and make sure you're actually using the code from them both before proceeding - if you can't test your changes, you'll have strange unsolvable problems later.

Branches

Each change submitted upstream should be in a separate branch. This makes it easy to review and modify the changes, so it helps everyone. As a rule, one PR means one branch.

git branch fix_spelling_error

This creates a new branch. You can check that your newly created branch exists by typing

Code: Select all

git branch

without options. You should see a * sign in front of the active branch, and notice that it isn't set to the new branch, so you need to switch (checkout) the newly created branch first, before you can work in it:

Code: Select all

git checkout fix_spelling_error

Now we're ready to make changes. Go and edit a file like lua_apl.txt or game_api.txt and make a small text change. Use a text editor and don't forget to save your changes.

Before you can submit your changes, you'll need to commit them to the repository as a changeset. This makes the modifications to the file part of the recorded history, and it includes writing a description, and summary. Your name is also recorded as part of the history.

To see what changes you have that are not committed yet, type:

Code: Select all

git diff

You should see a diff that displays the file name where you made changes, in a format that shows you easily to see what the part looked before, and after your change.

If the output looks correct, you can try to commit your changes. First, we tell git what files we would like to commit by staging the changes in the file for commit:


git add lua_api.txt


Now your change is staged, and you could stage more files. Or edit the file again and stage it again. And then finally:


git commit


Note: at this point you're probably going to get a warning from git about git not knowing what your name and e-mail address are, if you're doing this for the first time ever. You can tell git who you are by doing the two following commands:

Code: Select all

git config --global user.name "John Doe"
git config --global user.email "john.doe@example.com"

Once you do that, and retry the git commit, you should get to the commit message.

Commit message

I'm going to include a rather blunt statement here that is critical to getting your changes even considered, let alone included. So please take this with all seriousness and open-mindness at heart.

At best, your change, is a worthless piece of junk. It's going to break everything. It's going to offend people. It's going to hurt imaginary kittens (or puppies). And it's going to make everyone forget they had evolved to higher level humans and revert to levels of communication that certain species had mastered in the early Silurian.

For this reason, alone, you can not, ever, think that writing a commit message is less important than writing the code change that you had written. It is in fact, the most important part of your change.

You should take the act of writing a commit message and consider it the crowning advertisement of your change. You should write it in full sentences. You should write it in a readable fashion, and you should be dead honest about it while you do. If your change makes only a marginal improvement, be honest and acknowledge it.

So, start with the Subject of the commit message. Write something in 50 characters or so that describes the change so that it can be recognized, but not something that explains it.

Bad choices for subjects are:

"beds"
"more fixes"
"stuff"
"change this member to be public to avoid conflicts with std::scream operator in ElementIteratorFunction, because C++11 standard does not make opaque frequencies denunciated for unknown reasons"

Good choices for subjects are:

"Make bed placement not overwrite blocks"
"Prevent crash in on_metadata_*"
"API change documentation for schematics"
"Avoid C++11 compatibility in iterators"

As you can see, it doesn't need to be long. Longer probably means that you are trying to explain the change, but this goes in the summary.

The Summary should describe, explain, and motivate the change, but not as to such depth that it copies what the code change does. Let's dive into each of these 3 elements:

describing the change is an extended version of the summary. You use it to tell readers and reviewers where your changes are in the code, when they would be affecting the program, and how extensive they are.

explain your change where needed. Often this isn't needed, but sometimes it may not be obvious, and so then you'd explain exactly what is changed, and why the change achieves the effect required.

Motivate your change by explaining that currently something is broken, or is badly optimized, or just ugly or misspelled. Also describe how your patch actually solves that problem, too.

All done? On to making a PR then:

First, we need to push our change to our fork of the project:

Code: Select all

git push origin fix_spelling_error:fix_spelling_error

This creates a new remote branch in your fork on github with the same name as the local branch name. It also pushes the changes to this branch.

Now you can browse to github and go to your fork of the project, and you should see a "Create Pull Request" button. Click on it and follow the instructions.

Need to redo a change? Easy, just change the file again, but this time:

git add <file changed>
git commit --amend

and now your existing commit will be modified. Alternatively, you can look at git rebase -i if you want to do more cool things with combining commits.

Once you're happy with your committed changes, and tested them, you can push your changes again to the same branch:

Code: Select all

git push origin fix_spelling_error:fix_spelling_error --force

Because the remote branch has the old change, we need to overwrite the change, hence the "force" option.

The PR will automatically see your new changes, but you should visit it and check it, and leave a comment that you have changed it so people can revisit and re-review the change.

[sofar]

Covers topics needed to make working with Pull Requests fluid. Explains things like squashing and rebasing.

Once your PR is pending in github, any of the comments on it are added to the PR. We refer to them by number often, but each project has their own numbering. The PR itself is linked to your branch and to the target branch.

Because this link can not be changed, you can use this feature to do several things:

1. You can add more commits to the PR
2. You can change the commits in the PR (overwrite, and remove even!)
3. You can rebase the PR, or squash (essentially the same as the two above)

Adding more changes to an existing PR

Adding more changes is fairly trivial. In essence you just make more changes, commit them, and then just push them to the same remote branch of your fork! Because they are incremental to the previous change(s) you put in the PR, they will be added.

Changing the commits in an existing PR

First, you'll have to change your commit locally. Let's say that you modified lua_api.txt and want to fix up a typo you made, but not a new PR. You edit the file as you had left it after your commit (and submitted PR) in the same branch:

Code: Select all

git checkout mychangesbranch
git log  # check that you're on the right branch!
vi doc/lua_api.txt

Now, at this point you can make a new commit, but there are ways to get your change moved into the existing commit:

Code: Select all

git add doc/lua_api.txt
git commit --amend

This pops up the commit message editor with the last commit message, and now the new change is combined with the last commit!

From there on, all we need to do is push these changes to the PR branch on your fork:

Code: Select all

git push origin mychangesbranch:mychangesbranch -f

the -f flag tells git to force-push, which removes the old copy of your changes and overwrites it with the new version of your changes. Now you're done. You can revisit the github PR thread page and see your new changes at the bottom of the PR thread.


Squashing and Rebasing

Scared yet?

There's nothing magical in rebasing or squashing, but let's get the terminology right:

squashing means to combine several changes into one change that encompasses them all.
Rebasing means to take your changes, and to re-arrange them on top of other changes that happened elsewhere.

These might seem completely different things but we can use the same methods to do both of these. I'll highlight the easiest methods to do these:

Squashing

One of the easiest ways to squash is to do an interactive rebase. First we need to know how many commits to squash. Let's say we have the following git history in a branch:

Code: Select all

bd9c6e702331db6d5fdc5ce1e0aef436ecbb1111 Fire: move fire node removal out of ABM.
876cf8e5b60922a42a0c652d5c9e7c3af4b10c25 Coalblock: Use it as a fireplace, or beacon, etc.

The top one is the oldest here, the bottom one the newer. Note we have 2 commits here that we want to squash.

Type:

Code: Select all

git rebase -i HEAD~2

This means: do a rebase that is interactive and apply it to the last 2 commits.

This opens the editor with the following content:

pick bd9c6e7 Fire: move fire node removal out of ABM.
pick 876cf8e Coalblock: Use it as a fireplace, or beacon, etc.

Reading the help in the editor window, we can see that we can now squash a commit with a commit above it by changing the lines to look as follows:

pick bd9c6e7 Fire: move fire node removal out of ABM.
squash 876cf8e Coalblock: Use it as a fireplace, or beacon, etc.

Write the file and, voila, the two changes are now combined (and you get to edit the commit message in return!)

Rebasing

If you were asked to rebase, it's likely that changes were made to the same code upstream by someone else, and you now have to change your patches to account for those.

The basic method is just to run something like this:

Code: Select all

git pull --rebase upstream master

And sometimes this just works, and sometimes you'll get the dreaded merge conflict. I'd be a fool if I didn't point to several awesome documents that are available that already walk you through the steps. Here's a good github help page on the issue and how to work through conflicts:

https://help.github.com/articles/resolv ... mand-line/

[sofar]

tbd

[sofar]

I'm intending to write one of these topics every day in the next few days. If you would like to propose text, please post a reply and let me know in which section it should go!

[rubenwardy]

Looks good! Git is one of those things that I hate trying to explain

[afflatus]

Great thread sofar.

The most pressing issue for me is the nitty-gritty of how to submit a good clean PR the Minetest way.

[stu]

Perhaps a few words on pull request etiquette wouldn't go amiss :p

[sofar]

Perhaps a few words on pull request etiquette wouldn't go amiss :p

That's for the next part ;)

(yes yes I've been slacking off on another MT project for a sec)

[afflatus]

Not sure if this is within scope of this thread:

I have just been informed that github doesn't include submodules in the .zip downloads. So I'm looking for a quick and dirty way of removing submodules from my git master branch - would it work to simply delete .gitmodules and the .git dirs from the relevant sub dirs, then push it back out?

I have no desire for the learning curve involved in switching to git subtree right now. Although that might be a good subject for a future tutorial.

EDIT: [solved] git rm --cached <path/to/submodule>; git commit -a; git push ...
Seems to have done the trick.

[sofar]

Part 3 is now written. It's fairly complete I think. Please comment if you spot errors or omissions!

[afflatus]

That is the clearest explanation I have ever read.
Once we have April 1st out of the way I will put it to the test.
I have a fair few things that I think are worth backporting from grailtest. :)

[sofar]

I'm going to change item 4 into a topic covering Pull Request manipulations. I hope I can write it relatively soon.

[kaeza]

Looks good! Git is one of those things that I hate trying to explain

Obligatory

[sofar]

I've written section 4: changing/updating PR's and squashing and rebasing!

[benrob0329]

Could you please add a section explaining how to use vim to make a pull request?

(Like http://stackoverflow.com/questions/4164 ... -confusion)

[Krock]

That's a huge amount of information. This surely will help to get into Git and its features easily for a beginner.
Did you think about adding the tutorial on the Minetest wiki? It would make it easier to write the chapters and to do cross-references.

[sofar]

Could you please add a section explaining how to use vim to make a pull request?

(Like http://stackoverflow.com/questions/4164 ... -confusion)

I'm a bit hesitant to make it an editor HOWTO...

[sofar]

That's a huge amount of information. This surely will help to get into Git and its features easily for a beginner.
Did you think about adding the tutorial on the Minetest wiki? It would make it easier to write the chapters and to do cross-references.

Yes, this can certainly go in there. In only recently got a wiki account, I'll have to take a look at that first.

[benrob0329]

"Press A to go into insert mode, ESC to exit insert mode, to exit type ':wq' " would be very helpful.

[sofar]

"Press A to go into insert mode, ESC to exit insert mode, to exit type ':wq' " would be very helpful.

I'd rather include a link to a basic VIM tutorial instead...

[benrob0329]

How do I merge a pull request locally?

[sofar]

How do I merge a pull request locally?

Code: Select all

git pull git@github.com:pullrequestmaker/minetest pullrequestbranch

is one way to do it. You can find out what `pullrequestmaker` and `pullrequestbranch` are by looking at the top of the pull request github page. Change minetest to minetest_game where needed.