Recovering lost files after a git reset –hard

This morning I’ve messed things up in my env…

I did a commit yesterday and thought I had pushed it, but not.
Today I did a git pull and was asked to do a merge, because there were changes in the remote.

I totally ignored this (don’t ask me why, perhaps absence of coffee…) and did a git reset –hard HEAD^^

So, yep, I lost my commit that existed only in my machine (this commit in particular was a file addition).

After blaming myself a lot, and a bit of google search, I’ve found this amazing blog post: http://zsoltfabok.com/blog/2012/06/recover-deleted-files-with-git/

I’m summarizing here the steps to have your commit (in my case, the file) back:

git fsck --no-reflog

This command will return a hash for the lost commit, something like

dangling commit fafeade9f348b18f37835ab49dab40172efde693

You can use this hash to do a checkout, for instance, and recover your file. \o/

So, let’s have a bit of coffee now :)

Moving on

[ENGLISH]

Hi guys, I’d like to share with you the news that my family and some friends already know: Since last month I’m not working at Intel anymore.

For those whom don’t know my history, I’ll summarize it here:

  • About 3 years ago I left my home city to the biggest one in Brazil (São Paulo) to experience a full time Linux job
  • The instability of my department at the time led me to look for another job; Then I got that opportunity at Intel
  • Intel OTC office is located in Campinas (120km far away from São Paulo)
  • After more than one year having to travel about 240km every day I had to make a decision: Move to Campinas or leave Intel.

It was not an easy decision. I have a family and I didn’t want to affect them again with a city change. On the other side, Intel is a great company and people at OTC Campinas are amazing.

So, two weeks ago I started working for an IT Security company, working again with Embedded Linux and with Open Source technologies. Oh, 8km (15 min) far from my home :).

That’s it, I hope to blog a bit more now, as I used to do in the past. See you!

 [PORTUGUÊS]

Oi gente, gostaria de compartilhar uma novidade com vocês, que minha família e alguns amigos já sabem: Desde o mês passado que eu não trabalho mais na Intel.

Pra quem não conhece minha história, vou resumir aqui:

  • Cerca de 3 anos atrás eu mudei de Maceió, minha cidade natal pra São Paulo, pra trabalhar exclusivamente com Linux
  • Alguns problemas no meu departamento me levaram a buscar outro trabalho; Aí que entra a Intel
  • O escritório onde fui alocado (OTC, Open Source Technology Center) fica em Campinas, distante 120km de São Paulo
  • Depois de mais de 1 ano viajando cerca de 240km todos os dias eu tinha que tomar uma decisão: Mudar pra Campinas ou sair da Intel

Com certeza não foi uma decisão fácil. Tenho uma família e não queria afetá-los com mais uma mudança de cidade. Por outro lado, a Intel é uma ótima empresa e o pessoal do OTC Campinas é ótimo.

Então… duas semanas atrás comecei a trabalhar para uma empresa de segurança em TI, voltei a trabalhar com Linux embarcado e com Open Source de fato. Ah, e a 8km (15min) de casa :).

É isso, gente. Espero voltar a blogar com mais frequencia, como no passado. Até mais!

Parasite, wake up!

Hi there!

In the last weeks I’ve been working on GTK+ themes, customizing some apps, and I missed the tools we have on the web world that allow us to to some live editing on CSS.

I remembered we had Parasite, a tool similar to Firebug, but for GTK+ apps. Unfortunately it didn’t support CSS tweaking. But it does now! I’ve made some improvements on it mainly to allow live editing of CSS, but not limited to that.

Some changes so far:

  • Live editing of CSS, for the whole application, or only for the selected widget
  • Ability to add/remove a CSS class for a specific widget. (Editing of pseudo-classes like :hover is planned)
  • In the Property inspector, if a property itself is another Object, we can also inspect it. (For example, you can inspect a Buffer inside a TextView)
  • A bit of UI change

I have made a small video showing the improvements:


Link to the video

The code is on the usual place, github: https://github.com/chipx86/gtkparasite

Please, test it and file bugs/wishes/pull requests. /me is now wearing the maintainer hat :)

Note: it requires GTK+ 3.10 (and optionally python+gobject introspection for the python shell)

Joining Intel

Today is my last day at Oi WiFi.

It has been 1 year and a half since I moved from my small city (Maceió) to the biggest, craziest Brazilian city, São Paulo. I don’t regret!

I’m lucky to have joined a great company (Vex at the time. Oi WiFi nowadays), with great people where I learnt a lot. I’m glad for the things I helped to improve, I’m sure we have better products than before and I’m proud to be part of that progress. I leave as legacy the spirit of the Free Software, where we can (and should) contribute back to projects we use and improve internally. Every improvement we made here we submitted back to projects like Openwrt, busybox, glib, etc.
However things and priorities in the company have changed a bit in the last few months. Time to look for a new challenge in my career.

What a challenge!

At Intel I’ll join the OTC – Intel Open Source Technology Center, and will work on Open Source projects such as Tizen, EFL, Webkit and hopefully GTK+ :)
The team I’ll work with is formed by the former Profusion company, acquired by Intel in the beginning of the year. Profusion was a company that I admired even before it was acquired by Intel :)

I’m very excited to join Intel. It’s a great opportunity in a great company and I don’t want to disappoint them!

I hope to publish here very soon the things I’m working on under the Intel umbrella. See you!

Shell script that updates itself

Recently I needed to write a shell script that updates itself, and, surprising, I found it an easy job to do. I will share the recipe here.

In my use case, I’m developing a kind of software updater and, before updating the system packages, I need to check if there is a new version of this software updater. If there is, then I update myself and run my new copy on-the-fly.

Enough talk, show me the code. I’ll paste here a simple shell script that talks by itself:

#!/bin/sh

SCRIPT_NAME="$0"
ARGS="$@"
NEW_FILE="/tmp/blog.sh"
VERSION="1.0"

check_upgrade () {

  # check if there is a new version of this file
  # here, hypothetically we check if a file exists in the disk.
  # it could be an apt/yum check or whatever...
  [ -f "$NEW_FILE" ] && {

    # install a new version of this file or package
    # again, in this example, this is done by just copying the new file
    echo "Found a new version of me, updating myself..."
    cp "$NEW_FILE" "$SCRIPT_NAME"
    rm -f "$NEW_FILE"

    # note that at this point this file was overwritten in the disk
    # now run this very own file, in its new version!
    echo "Running the new version..."
    $SCRIPT_NAME $ARGS

    # now exit this old instance
    exit 0
  }

  echo "I'm VERSION $VERSION, already the latest version."
}

main () {
  # main script stuff
  echo "Hello World! I'm the version $VERSION of the script"
}

check_upgrade
main

To try this script:
1) save it somewhere
2) save a copy of it at /tmp/blog.sh (as pointed at line 5)
3) modify the variable “VERSION” (line 6) of that copy, to, say, “2.0”.
4) run the original script (not the one at /tmp)

You will see that the script updated itself and ran the “new” 2.0 version.

Try running again the original script (step 4 above). See the difference? It doesn’t update itself anymore, because it is the “latest” version.

A small thing you might notice: at line 19, I deleted the “new file”. That’s merely for this educational example, that we check if there’s a new version of the script by just checking if a file exists in the disk. On real life (with apt/yum or any smarter process) this is not needed as our check for a new version (line 13) will naturally fail.

This was tested with bash, dash and busybox’s ash. Worked fine.

I hope it’s useful to someone. Comments are welcome!

g_list_copy_deep and g_slist_copy_deep in GLib

If you have in your code things like that:

copy = g_slist_copy (list);
g_slist_foreach (copy, g_object_ref, NULL);

In GLib 2.34 (to be released soon) you can do:

copy = g_slist_copy_deep (list, g_object_ref, NULL);

Thus eliminating another loop in the list (one loop is done when copying the list).

In other words, there’s now a function to do a deep (full) copy of a GList and GSList. It accepts, besides the list to be cloned, a function as a second argument, and a user data as the third argument. That function will be called for each element in the list.

To free the new list, use g_list_free_full().

Programmatically managing iptables rules in C: IPTC

Sometimes we need to manage iptables rules from inside our own programs. The recommended solution given by iptables developers is to spawn the iptables command with execl() or system(). It’s explicitly stated that there’s no stable/public API to do that: http://www.netfilter.org/documentation/FAQ/netfilter-faq-4.html#ss4.5

However, if you have control of your box, including kernel, iptables, and all related packages, it’s safe to use IPTC to programmatically manage iptables rules.

IPTC is an internal iptables library that actually inserts/removes the rules. It’s used by the iptables (xtables-multi, iptables-save, iptables-restore) binaries. Depending on how iptables was compiled, there might be a shared libiptc.so object and/or a static libiptc.a object.

You should check that in order to correctly build your application. In Debian/Ubuntu, which this post is based, it’s enough to install the iptables-dev package. It will install the shared libraries, the header files and the pkg-config facility.

So, let’s take a look at a complete function that adds a rule to iptables:

#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <libiptc/libiptc.h>

static int
insert_rule (const char *table,
             const char *chain, 
             unsigned int src,
             int inverted_src,
             unsigned int dest,
             int inverted_dst,
             const char *target)
{
  struct
    {
      struct ipt_entry entry;
      struct xt_standard_target target;
    } entry;
  struct xtc_handle *h;
  int ret = 1;

  h = iptc_init (table);
  if (!h)
    {
      fprintf (stderr, "Could not init IPTC library: %s\n", iptc_strerror (errno));
      goto out;
    }

  memset (&entry, 0, sizeof (entry));

  /* target */
  entry.target.target.u.user.target_size = XT_ALIGN (sizeof (struct xt_standard_target));
  strncpy (entry.target.target.u.user.name, target, sizeof (entry.target.target.u.user.name));

  /* entry */
  entry.entry.target_offset = sizeof (struct ipt_entry);
  entry.entry.next_offset = entry.entry.target_offset + entry.target.target.u.user.target_size;
  
  if (src)
    {
      entry.entry.ip.src.s_addr  = src;
      entry.entry.ip.smsk.s_addr = 0xFFFFFFFF;
      if (inverted_src)
        entry.entry.ip.invflags |= IPT_INV_SRCIP;
    }

  if (dest)
    {
      entry.entry.ip.dst.s_addr  = dest;
      entry.entry.ip.dmsk.s_addr = 0xFFFFFFFF;
      if (inverted_dst)
        entry.entry.ip.invflags |= IPT_INV_DSTIP;
    }

  if (!iptc_append_entry (chain, (struct ipt_entry *) &entry, h))
    {
      fprintf (stderr, "Could not insert a rule in iptables (table %s): %s\n", table, iptc_strerror (errno));
      goto out;
    }

  if (!iptc_commit (h))
    {
      fprintf (stderr, "Could not commit changes in iptables (table %s): %s\n", table, iptc_strerror (errno));
      goto out;
    }

  ret = 0;
out:
  if (h)
    iptc_free (h);

  return ret;
}

int main (int argc, char **argv)
{
  unsigned int a, b;

  inet_pton (AF_INET, "1.2.3.4", &a);
  inet_pton (AF_INET, "4.3.2.1", &b);

  insert_rule ("filter",
               "INPUT",
               a,
               0,
               b,
               1,
               "DROP");
  return 0;
}

Note: The code above was compiled and tested against the 1.4.13 version of iptables.
To compile:

cc `pkg-config –cflags –libs libiptc` source-file.c -o test-iptc

So, the code in main() above does the equivalent of:

iptables -t filter -A INPUT -s 1.2.3.4/32 ! -d 4.3.2.1/32 -j DROP

The main functions used were iptc_init, iptc_append_entry and iptc_commit. There are functions to remove a rule (iptc_delete_num_entry), to iterate over all the rules (iptc_first_rule, iptc_next_rule) and to clear all rules (iptc_flush_entries).

You can check the header file libiptc/libiptc.h to see all available functions and their descriptions. I can post some more examples here in the future if there’s demand to it.

Again, remember that there’s no API stability in IPTC, so, it can change in each new version of iptables.

Hope it’s useful to someone. Feel free to ask anything or to report any issues you have. See you!

Linux firewall: Mac filtering – iptables and arptables|Firewall no Linux: filtro por mac – iptables e arptables

[english]

It’s common: you don’t want just filter the access by IP address, but also by the MAC address. The solution can be quite simple: use the MAC module of iptables:

iptables -A INPUT -s 1.2.3.4 -m mac ! –mac-source aa:bb:cc:dd:ee:ff -j DROP


The command above says: Drop every packet with source ip address 1.2.3.4 and its MAC address is NOT aa:bb:cc:dd:ee:ff. Thus you’re forcing the origin of the packet to be the desired one.


However there is a minor issue with that approach: iptables manages packets at the network (IP) layer, which is above the link (MAC) layer. What does that mean? Simple: Once the packet arrives at the network interface, it’s handled by the MAC layer and then it’s forwarded to the IP layer. That’s where iptables filter it. Again, what’s the problem? At this pointer, the ARP table of the kernel was touched, meaning that (using the example above), if a packet with source IP 1.2.3.4 and source MAC 00:00:00:00:00:01 arrives, it will change the ARP entry in the kernel table to point the IP 1.2.3.4 to that wrong MAC address. After that, the packet is forwarded to the above layer, which is then dropped by iptables from go on over the network. Even if the packet is dropped, which is what you want, your ARP table is messed up right now.


That’s where arptables command comes in. It works just like iptables, but it filters packets at MAC layer, preventing the kernel ARP table being touched by wrong packets. Its syntax is very similar to iptables (in fact, it’s a copy of iptables adapted to work in the lower level). The same command (iptables) above could be better implemented with arptables:

arptables -A IN -s 1.2.3.4 ! –source-hw aa:bb:cc:dd:ee:ff -j DROP


You just need this rule, it’s not necessary to have the iptables rules anymore. arptables needs a kernel with support built in, which I guess most distros do, just like they do with iptables.


Take a look at the arptables man page, it has some examples. Its similarity with iptables facilitates our lives.

[português]

É uma situação comum: você não quer apenas filtrar o acesso por endereço IP, mas também pelo endereço MAC. A solução pode ser bastante simples: usar o módulo MAC do iptables:

iptables -A INPUT -s 1.2.3.4 -m mac ! –mac-source aa:bb:cc:dd:ee:ff -j DROP


O comando acima diz: Descarte todos os pacotes com o endereço IP de origem 1.2.3.4 e cujo endereço MAC não é aa:bb:cc:dd:ee:ff. Assim, você está forçando a origem do pacote a ser a desejada.


No entanto, há um pequeno problema com essa abordagem: o iptables gerencia pacotes na camada de rede (IP), que é acima da camada de enlace (MAC). O que significa isso? Simples: Quando o pacote chega na interface da rede, ele é tratado pela camada MAC e depois é encaminhado para a camada IP. É onde iptables o filtra. Novamente, qual é o problema? Neste ponto, a tabela ARP do kernel foi alterada, o que significa que (usando o exemplo acima), se um pacote com o IP origem 1.2.3.4 e MAC 00:00:00:00:00:01 chega, ele vai mudar a entrada na tabela ARP do kernel para apontar o IP 1.2.3.4 para o endereço MAC errado. Depois disso, o pacote é encaminhado para a camada superior, que é então descartado pelo iptables de continuar na rede. Mesmo o pacote sendo descartado, que é o que você deseja, sua tabela ARP está uma bagunça agora.


É aí que o comando arptables entra. Ele funciona exatamente como o iptables, mas filtra os pacotes na camada MAC, evitando que a tabela ARP do kernel seja alterada por pacotes errados. Sua sintaxe é muito semelhante ao iptables (na verdade, é uma cópia do iptables adaptado para trabalhar num nível mais baixo). O mesmo comando (iptables) acima poderia ser melhor implementado com arptables:

arptables -A IN -s 1.2.3.4 ! –source-hw aa:bb:cc:dd:ee:ff -j DROP


Você só precisa dessa regra, não é necessário ter as regras de iptables mais. o arptables precisa que o kernel tenha suporte, o que eu acho que a maioria das distribuições já fazem, assim como eles fazem com o iptables.


Dê uma olhada no man do arptables, lá tem alguns exemplos. Sua semelhança com o iptables facilita nossas vidas.

free() inconsistencies|inconsistências no free()


Hi, folks!


Last week I was porting a program from uClibc to glibc. Everything went fine, until I found a crash to always happen in a certain part of it. The failure was in a call to the free() function. First thing that came in my head: Why the hell is it crashing now, while it used to run fine on uClibc? I made a simple program that simulates the problem:


Oi gente!


Semana passada tava portando um código compilado em cima da uClibc para glibc. Tudo tranquilo, até que certa parte do programa dava crash. Investigando, vi que a falha acontecia em uma chamada à função free(). Primeira coisa que veio à minha cabeça: porque raios tá dando crash aqui se o mesmo programa, compilado na uClibc roda perfeito? Fiz um programa simples que simula a situação:

#include &lt;stdio.h&gt;
#include &lt;stdlib.h&gt;

typedef struct {
  char *field1;
} s_test;

s_test test = {
  .field1 = NULL
};

int main (int argc, char **argv) {
  s_test *t;

  t = &amp;test;
  free (t);
  t = &amp;test;
  t-&gt;field1 = "bug";
  printf ("%s\n", t-&gt;field1);

  return 0;
}


Look at line 16. I’m executing a free() in a pointer to a static variable, instead of a pointer in the heap (previously allocated with malloc() or similar). It’s expected a crash here, right? Maybe! Yes, if you’re using glibc. No if you’re using uClibc. The above code works like [not] expected. Weird! Everything we learned at the programming school is ruined now :D !


So, we have a similar code here that have been worked for a long time, exactly because it was compiled and run on top of uClibc. I’ve seen this and other behavior differences between uClibc and glibc. The solution? Change the code to make it portable, not only to make it compile, but also so that it have the same behavior on every platform.


I thought it was a bug in uClibc, but I was told it doesn’t break the standards. In fact, standards say, in that case, the behavior is “undefined”. Ah, standards :) … So, in order to avoid surprises like that, here is what I learned: Always code in the right way, even if it comes with a harder job. Don’t say: “hey, it’s working, let’s deploy it!”.


See you!


Preste atenção na linha 16. Estou executando um free() num ponteiro que aponta para uma variável estática, ao invés de uma variável que foi alocada com malloc() ou similar. Um crash é esperado aqui, certo? Em partes! Usando a glibc, sim. Já com a uclibc, não! O código acima funciona como [não] esperado! Estranho, não? Tudo o que aprendemos na escola de programação vai por água abaixo aqui hehehe.


Então, o que acontece é que temos um código similar aqui e que sempre funcionou, justamente por ser compilado na uclibc. Já vi essa e outras diferenças de comportamento entre a uclibc e a glibc. A solução? Mudar o código para torná-lo portável, não só para que compile corretamente, mas para que tenha os mesmos resultados, independente da plataforma.


A princípio, achei que isso era um bug na uclibc, mas fui apontado que isso não fere “os padrões”. De fato, os padrões dizem que nesse caso, o comportamento é “indefinido”. Ah, padrões :) … Para evitar surpresas do tipo, fica aqui a lição aprendida: programar da forma certa, mesmo que dê um pouco mais de trabalho. Não se acomodar dizendo: “ah, testei aqui e funciona, deixa assim mesmo!” ;)


Bons códigos!

New challenges|Novos desafios

[english]


Hi, folks!


This is a small post, just to share the news with you: I moved to São Paulo (the biggest Brazilian city) to work with what I like the most: linux development! It’s indeed a big step in my career, I’m too excited about my new job: embedded linux developer!


I hope to learn a lot in this new job, and, at the right time, I want to contribute back to FLOSS community, since we use a lot of free software here.


See you!

[português]

Oi gente, tudo bom?

Este é um post rápido, quero compartilhar uma novidade com vocês: Estou dando um grande passo na minha carreira profissional. Me mudei para São Paulo e estou trabalhando na Vex/OI como desenvolvedor linux embarcado.

Foram pouco mais de 6 anos no meu último emprego, numa usina de açúcar e álcool em Alagoas. Fiz muitos amigos e aprendi bastante lá. Foi uma ótima empresa de se trabalhar. Infelizmente lá eu não trabalhava com o que eu gosto: linux. Por isso estou de mudança.

Espero aprender bastante nesse novo emprego e, no tempo certo, contribuir de volta com a comunidade FLOSS, visto que usamos muita tecnologia livre aqui.

Por enquanto estou só, procurando lugar pra morar. A família vem no final do ano. Quem quiser nos visitar em Sampa, pode entrar em contato, será um prazer receber os amigos!