Description:

The network connection to ibs from Beijing is quite slow. And ibs doesn't supply the search function of tags in spec files, like versions, requirements, or patches, etc...

Tasks to complete the projects:

  1. Spider the specs in ibs.
    • Already implemented a spider of another web via Python, so plan to choose golang without any 3rd party framework to fetch. To familiar with goroutine for parallel work, though threads may more effective since network should be the bottleneck so far...
  2. Save the results into database with tags like projects, package name, etc...
    • Not decided yet which database to pick, or even sql/nosql model... Current idea is mysql/memcached or redis, depends on how to compose the tables/key of database.
  3. Analyse and polish the code to make it more fast.
    • Try to make the local engineer become high availability cluster, low latency and higher throughput. Using Galera or Pacemaker for availability? nginx or haproxy for load balance? In memory database for accelerating the read? No idea at the moment, should depends on the bottleneck and the needs. Though a simple engineer for internal won't require much, but may learn more while improving/thinking about the questions.
  4. Check and compare the performance like TPS...
  5. A nice web page to show the result?
    • It is not the 1st priority task of me... let's see in future:)

The target of the project are:

  1. Learning and familiar with golang. Currently i have nearly 0 experience on golang for a real project:)
  2. Learn more while constantly questioning the system.
  3. Have fun.

I don't think the project can be finished within 1 week. Hopefully i can at least get something after that and continue the task when have spare time. First to do it and have fun:)

Looking for hackers with the skills:

Nothing? Add some keywords!

This project is part of:

Hack Week 17

Activity

  • over 5 years ago: hennevogel liked this project.
  • over 5 years ago: acho liked this project.
  • over 5 years ago: wanghaisu started this project.
  • over 5 years ago: wanghaisu originated this project.

  • Comments

    • wanghaisu
      over 5 years ago by wanghaisu | Reply

      Seems sub-list of mark down is not supported... at least i failed:(

      • wanghaisu
        over 5 years ago by wanghaisu | Reply

        works but not good as expect:(

        • hennevogel
          over 5 years ago by hennevogel | Reply

          Patches welcome add-emoji

          • wanghaisu
            over 5 years ago by wanghaisu | Reply

            I found it is a bug of redcarpet 3.4.0 which hackweek use, refer to issue redcarpet doesn't release a new version for more than 1 year:( Will see a better solution.

            • wanghaisu
              over 5 years ago by wanghaisu | Reply

              Solution for example like an extend markdown generater for CommonMark, which is GitHub flavored at least. Anyway, i need to setup a hackweek locally to do a real test before any submission:)

    Similar Projects

    This project is one of its kind!