building care-o-bot 79867c8 #262

hsd-dev · 2019-03-13T16:58:29Z

While building the bug, I get the following error:

Updating local rosdistro index.yaml to use cache from the past ..
Invoking: rosinstall_generator --rosdistro=indigo cob_android --deps --tar --deps-only
Using ROS_DISTRO: indigo
The following not released packages/stacks will be ignored: cob_android
No packages/stacks left after ignoring not released
Traceback (most recent call last):
  File "scripts/build-rosinstall.py", line 57, in _time_machine
    stdout=subprocess.PIPE)
  File "/usr/lib/python3.5/subprocess.py", line 708, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['rosinstall_generator_tm.sh', '2015-08-03T08:03:34Z', 'indigo', 'cob_android', '--deps', '--tar', '--deps-only']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "scripts/build-rosinstall.py", line 173, in <module>
    main()
  File "scripts/build-rosinstall.py", line 166, in main
    build_file(fn_or_dir, overwrite=args.overwrite)
  File "scripts/build-rosinstall.py", line 118, in build_file
    deps.update(_time_machine(ros_pkgs, dt, distro, deps_only=True))
  File "scripts/build-rosinstall.py", line 61, in _time_machine
    err.returncode, fn_bug_desc)
NameError: name 'fn_bug_desc' is not defined

The issue was created at 2015-08-03T08:03:34Z and the package was first released on Aug 29, 2015. So basically, there is no entry of this package for the requested {TIMEPOINT} in rosdistro.

How can we solve this?

The text was updated successfully, but these errors were encountered:

git-afsantos · 2019-03-13T17:09:25Z

I think I ran into a similar issue once or twice before, and I think there is no alternative other than using the release date of the package. But perhaps, someone else has a better solution.

hsd-dev · 2019-03-15T15:31:52Z

The issue was already fixed before the 1st release.

gavanderhoorn · 2019-03-17T17:35:06Z

Related: rosin-project/rosinstall_generator_time_machine#11.

hsd-dev · 2019-03-20T12:51:00Z

Should I close this, since it is being tracked elsewhere?

gavanderhoorn · 2019-03-20T12:55:33Z

Well, the issue of repositories using source builds and reporting bugs in those is tracked in other issues.

But your problem reproducing 79867c8 is not.

What I would suggest is the following: generate a .rosinstall file using the first released version (to get all the dependencies in place) and then continue with the commit hash of the unfixed version (ie: the one that was not released).

That should work.

We should make a note about this (ie: not using the "correct" timepoint).

hsd-dev · 2019-03-20T15:05:21Z

I was able to generate deps.rosinstall from the following command:

rosinstall_generator_tm.sh 2015-08-30T08:03:34Z indigo cob_android --deps --deps-only --tar > deps.rosinstall

I guess now I have to modify the cob_android package entries manually to 79867c8.

gavanderhoorn · 2019-03-20T16:09:30Z

I guess now I have to modify the cob_android package entries manually to 79867c8.

~~yes, if that is the hash for the buggy state, then that would be the idea :)~~

Wait: since we use --deps-only, the PUT itself (cob_android in this case) will not appear in the .rosinstall file. I don't believe you need to do anything special now: the .bug should already contain the real hash.

hsd-dev · 2019-03-20T16:17:30Z

These are the cob_android package entries in the deps.rosinstall:

- tar:
    local-name: cob_android/cob_android_msgs
    uri: https://github.com/ipa320/cob_android-release/archive/release/indigo/cob_android_msgs/0.1.1-0.tar.gz
    version: cob_android-release-release-indigo-cob_android_msgs-0.1.1-0
- tar:
    local-name: cob_android/cob_android_resource_server
    uri: https://github.com/ipa320/cob_android-release/archive/release/indigo/cob_android_resource_server/0.1.1-0.tar.gz
    version: cob_android-release-release-indigo-cob_android_resource_server-0.1.1-0
- tar:
    local-name: cob_android/cob_android_script_server
    uri: https://github.com/ipa320/cob_android-release/archive/release/indigo/cob_android_script_server/0.1.1-0.tar.gz
    version: cob_android-release-release-indigo-cob_android_script_server-0.1.1-0
- tar:
    local-name: cob_android/cob_android_settings
    uri: https://github.com/ipa320/cob_android-release/archive/release/indigo/cob_android_settings/0.1.1-0.tar.gz
    version: cob_android-release-release-indigo-cob_android_settings-0.1.1-0

gavanderhoorn · 2019-03-20T16:22:38Z

cob_android is a metapackage, so cannot actually be the package any bug with install(..) statements is in.

Looking at the bug description, it would appear the affected packages are cob_android_msgs and cob_android_script_server (here). Those packages should be passed to the time machine, not the metapackage.

gavanderhoorn · 2019-03-20T16:23:38Z

Note: the bug description would seem to be incorrect. The bug is not in cob_android, but in those other packages. This line should probably be updated.

hsd-dev · 2019-03-21T12:05:24Z

Can someone give me permissions to fork repositories to this organization?

gavanderhoorn · 2019-03-21T12:09:07Z

No (as it's not a permission we can give to individual users), but we can invite you to the robust-rosin organisation. Which I just did.

hsd-dev · 2019-03-21T13:36:13Z

I don't remember if @ChrisTimperley said if bugzoo can handle multiple PUTs yesterday, but the bug file looks like this:

bug:
  package: cob_android_msgs cob_android_script_server
  issue: https://github.com/ipa320/cob_android/issues/7
  time-reported: 2015-08-03 (10:09)
time-machine:
  ros_distro: indigo
  ros_pkgs:
    - cob_android_msgs
    - cob_android_script_server
  issue: https://github.com/ipa320/cob_android/issues/7
bugzoo:
  fork-urls: https://github.com/robust-rosin/cob_android
  bug-commit: 21126f5e3d2e15ae8c93705dc3e8e999707da48e
  fix-commit: 79867c88acc64342f061e6324ed6b84ca7e0773d

And the output of

 python build-bugzoo.py

is

generating manifest for bug file [care-o-bot/79867c8/79867c8.bug]
build-bugzoo.py:71: UserWarning: BugZoo file does not currently support multiple PUTs
  warnings.warn('BugZoo file does not currently support multiple PUTs')

gavanderhoorn · 2019-03-21T13:37:49Z

IIRC, BugZoo: not tested (probably not). The .rosinstall script: most likely yes.

ChrisTimperley · 2019-03-26T02:17:30Z

generating manifest for bug file [care-o-bot/79867c8/79867c8.bug]
build-bugzoo.py:71: UserWarning: BugZoo file does not currently support multiple PUTs
  warnings.warn('BugZoo file does not currently support multiple PUTs')

#264 should add multiple PUT support to `build-bugzoo.py.

Also, fork-urls should be a list:

bugzoo:
  fork-urls:
    - https://github.com/robust-rosin/cob_android

hsd-dev · 2019-03-28T12:40:51Z

@ChrisTimperley I am able to build the manifest with #264. Thanks!

Now facing #69

gavanderhoorn · 2019-03-28T12:44:39Z

Now facing #69

#69? Can you show the build log for your bug?

Has the care-o-bot repository been renamed? Renames of geometry_experimental should be automatically handled.

Could it be that you're running into vcstools/vcstools#147?

hsd-dev · 2019-03-28T12:51:59Z

@gavanderhoorn : Sorry that was a hasty reply as I didn't dig into the error.

I am getting similar errors for tf2 related packages:

ERROR [vcstools] Tarball download unpack failed: u"geometry_experimental-release-release-indigo-tf2-0.5.12-0 is not a subdirectory with contents in members [u'geometry2-release-release-indigo-tf2-0.5.12-0']"[/vcstools]

ERROR in config: Error processing 'geometry_experimental/tf2' : [geometry_experimental/tf2] Checkout of https://github.com/ros-gbp/geometry_experimental-release/archive/release/indigo/tf2/0.5.12-0.tar.gz version geometry_experimental-release-release-indigo-tf2-0.5.12-0 into /ros_ws/src/geometry_experimental/tf2 failed.

gavanderhoorn · 2019-03-28T12:53:17Z

That would seem to be #69, but afaik we're already handling that with automatic renames of entries in .rosinstall files.

Can you clarify which versions of things you are using, and which commands lead to this error?

hsd-dev · 2019-03-28T13:14:35Z

I pulled #264 in a new branch and working on it.

These are the tars in deps.rosinstall which fail:

- tar:
    local-name: geometry_experimental/tf2
    uri: https://github.com/ros-gbp/geometry_experimental-release/archive/release/indigo/tf2/0.5.12-0.tar.gz
    version: geometry_experimental-release-release-indigo-tf2-0.5.12-0
- tar:
    local-name: geometry_experimental/tf2_msgs
    uri: https://github.com/ros-gbp/geometry_experimental-release/archive/release/indigo/tf2_msgs/0.5.12-0.tar.gz
    version: geometry_experimental-release-release-indigo-tf2_msgs-0.5.12-0
- tar:
    local-name: geometry_experimental/tf2_py
    uri: https://github.com/ros-gbp/geometry_experimental-release/archive/release/indigo/tf2_py/0.5.12-0.tar.gz
    version: geometry_experimental-release-release-indigo-tf2_py-0.5.12-0
- tar:
    local-name: geometry_experimental/tf2_ros
    uri: https://github.com/ros-gbp/geometry_experimental-release/archive/release/indigo/tf2_ros/0.5.12-0.tar.gz
    version: geometry_experimental-release-release-indigo-tf2_ros-0.5.12-0

The following command lead to the error:

bugzoo bug build --force robust:79867c8

gavanderhoorn · 2019-03-28T13:16:47Z

Renaming happens when the .rosinstall file is generated (here), but only if you actually used scripts/build-rosinstall.py.

Did you run the time machine manually?

Ah, just saw that you did: #262 (comment).

That would explain it then.

Either rename the entries yourself (to geometry2) or use scripts/build-rosinstall.py.

hsd-dev · 2019-03-28T13:20:15Z

Ah, just saw that you did: #262 (comment).

Was about to comment that :)

Can I use scripts/build-rosinstall.py when I am selecting the {TIMEPOINT} manually?

gavanderhoorn · 2019-03-28T13:23:57Z

Can I use scripts/build-rosinstall.py when I am selecting the {TIMEPOINT} manually?

Well .. no, not really (if I understand you correctly).

scripts/build-rosinstall.py will use the time-machine/datetime key in the .bug description to drive the time machine. In this case I would just manually replace geometry_experimental with geometry2 and commit the updated .rosinstall.

Note that you may still run into vcstools/vcstools#147 unfortunately.

hsd-dev · 2019-03-28T13:55:04Z

Manually replacing geometry_experimental with geometry2 helps.

But docker build in

bugzoo bug build --force robust:79867c8

still fails with

/ros_ws/src/cob_driver/cob_sound/ros/src/sound.cpp:9:32: fatal error: cob_srvs/SetString.h: No such file or directory
 #include <cob_srvs/SetString.h>
                                ^
compilation terminated.

make[2]: *** [CMakeFiles/sound.dir/ros/src/sound.cpp.o] Error 1

make[1]: *** [CMakeFiles/sound.dir/all] Error 2

make: *** [all] Error 2

Command failed, exiting.

Looks like cob_sound-0.6.4 (which is currently listed in deps.rosintall) did not include cob_common as its dependency. It was done in cob_sound-0.6.5

gavanderhoorn · 2019-03-28T13:56:44Z

That would be a candidate for missing-dependencies then. See #150.

Edit: example:

robust/mavros/753226d/753226d.bug

Lines 55 to 56 in a293039

    
           missing-dependencies: 
        
             - mavros_extras

hsd-dev · 2019-03-28T14:30:12Z

But don't I have to use scripts/build-rosinstall.py for missing-dependencies as well?
I might have to manually change the release version in this case.

gavanderhoorn · 2019-03-28T14:34:39Z

But don't I have to use scripts/build-rosinstall.py for missing-dependencies as well?

If you want to have everything done automatically: yes.

But in the end all it does is run the time machine twice. Once for the actual PUT(s) (and with --deps-only) and once for the missing dependencies (without --deps-only). See also ~~#150~~ #254.

I might have to manually change the release version in this case.

Don't believe that's needed in this case.

As this is a special case (bug in unreleased pkg) I'd just run the time machine manually, as you're already doing.

Edit: this part may be clearer. #254 is a bit difficult to follow.

hsd-dev · 2019-03-28T15:40:44Z

in scripts/build-rosinstall.py

        deps.update(_time_machine(ros_pkgs, dt, distro, deps_only=True))
        if missing_deps:
            deps.update(_time_machine(missing_deps, dt, distro, deps_only=False))

Makes sense.

Ran the command

rosinstall_generator_tm.sh 2015-09-01T08:03:34Z indigo cob_common --deps --tar > deps.rosinstall

deps.rosintall looks file now.

But #264 (comment)

EDIT: Sorry my bad! All good now :)

hsd-dev · 2019-04-03T07:18:54Z

Guess I was unclear, but I am able to build the image now.

gavanderhoorn · 2019-04-03T07:21:44Z

@ipa-hsd: with #270 merged, you should be able to set the timestamp you used for the .rosinstall files (ie: the one after the first release) in the .bug file, and Bugzoo should make the time machine use that instead of the timestamp of the issue (which would be too early).

It would be great if you could test whether that actually works.

hsd-dev · 2019-04-03T10:39:14Z

I merged the master branch (which should include #270) and rebased #264

I am able to build the image normally now.

gavanderhoorn · 2019-04-03T10:40:16Z

Which timestamp did you configure for the time machine to use?

hsd-dev · 2019-04-03T10:44:47Z

2015-08-30T08:03:34Z, which is 1 date after the 1st release of cob_android

gavanderhoorn · 2019-04-04T10:20:57Z

@ipa-hsd: perhaps you've already done so, but could I ask you to add a comment to the .bug file explaining that timestamp (ie: that it's a datetime later than the original reporting time of the issue, due to the package not being released, etc)?

hsd-dev · 2019-04-04T10:29:50Z

I hadn't, will do so 👍

ChrisTimperley mentioned this issue Mar 26, 2019

Added support for multiple PUTs (assuming a single fork) #264

Merged

gavanderhoorn mentioned this issue Mar 28, 2019

build-rosinstall: should it use datetime before using issue url? #269

Closed

hsd-dev closed this as completed Apr 3, 2019

hsd-dev reopened this Apr 3, 2019

hsd-dev closed this as completed Apr 4, 2019

gavanderhoorn mentioned this issue May 8, 2019

ur_kinematics not released #275

Closed

building care-o-bot 79867c8 #262

building care-o-bot 79867c8 #262

Comments

hsd-dev commented Mar 13, 2019

git-afsantos commented Mar 13, 2019

hsd-dev commented Mar 15, 2019

gavanderhoorn commented Mar 17, 2019

hsd-dev commented Mar 20, 2019

gavanderhoorn commented Mar 20, 2019 • edited Loading

hsd-dev commented Mar 20, 2019

gavanderhoorn commented Mar 20, 2019 • edited Loading

hsd-dev commented Mar 20, 2019

gavanderhoorn commented Mar 20, 2019

gavanderhoorn commented Mar 20, 2019

hsd-dev commented Mar 21, 2019

gavanderhoorn commented Mar 21, 2019

hsd-dev commented Mar 21, 2019

gavanderhoorn commented Mar 21, 2019

ChrisTimperley commented Mar 26, 2019

hsd-dev commented Mar 28, 2019

gavanderhoorn commented Mar 28, 2019

hsd-dev commented Mar 28, 2019

gavanderhoorn commented Mar 28, 2019

hsd-dev commented Mar 28, 2019

gavanderhoorn commented Mar 28, 2019 • edited Loading

hsd-dev commented Mar 28, 2019 • edited Loading

gavanderhoorn commented Mar 28, 2019 • edited Loading

hsd-dev commented Mar 28, 2019

gavanderhoorn commented Mar 28, 2019 • edited Loading

hsd-dev commented Mar 28, 2019

gavanderhoorn commented Mar 28, 2019 • edited Loading

hsd-dev commented Mar 28, 2019 • edited Loading

hsd-dev commented Apr 3, 2019

gavanderhoorn commented Apr 3, 2019

hsd-dev commented Apr 3, 2019 • edited Loading

gavanderhoorn commented Apr 3, 2019

hsd-dev commented Apr 3, 2019

gavanderhoorn commented Apr 4, 2019

hsd-dev commented Apr 4, 2019

gavanderhoorn commented Mar 20, 2019 •

edited

Loading

gavanderhoorn commented Mar 20, 2019 •

edited

Loading

gavanderhoorn commented Mar 28, 2019 •

edited

Loading

hsd-dev commented Mar 28, 2019 •

edited

Loading

gavanderhoorn commented Mar 28, 2019 •

edited

Loading

gavanderhoorn commented Mar 28, 2019 •

edited

Loading

gavanderhoorn commented Mar 28, 2019 •

edited

Loading

hsd-dev commented Mar 28, 2019 •

edited

Loading

hsd-dev commented Apr 3, 2019 •

edited

Loading