Jump to content

Mikael Kalms

  • Content count

  • Joined

  • Last visited

  • Days Won


Mikael Kalms last won the day on March 17 2017

Mikael Kalms had the most liked content!

Community Reputation

1 Neutral

About Mikael Kalms

  • Rank
    Advanced Member
  1. Jenkins Shared Libraries in Plastic SCM

    I am experimenting a bit with loading in the support modules using the 'load' command in Groovy. Problem: on the master, the "Pipeline script from SCM" will result in the repository being checked out to .../TestProject@script/PlasticWorkspaceName/<files> but when the Jenkinsfile is executed, the context (both ${WORKSPACE} and pwd()) point to the .../TestProject/, i.e. the root of the 'real' workspace checkouts on the slaves. I do not know how to phrase the load statement correctly on both master and slave.
  2. Jenkins Shared Libraries in Plastic SCM

    Ideally I would like to: * Have one single (large) repository with the source code for my product; this does not necessarily have to include the build configurations * Describe my build configurations using Declarative Pipeline syntax, and have this stored somewhere in source control * Be able to share common build logic between multiple build configurations, and have this shared logic also stored somewhere in source control Today I have a single Plastic repo that contains the product + 5 build configuration files (that is, 5 Jenkinsfiles). I have one Jenkins job configured for each build configuration, and I specify the appropriate Jenkinsfile in the Pipeline Job configuration. This is reasonable... except that I have a blob of Groovy support-functions at the top of each Jenkinsfile. I would like to be able to move out that blob of support functions to a couple of Groovy source files in a single, shared location. If I was using Git, I would move the Groovy support-functions to a separate Git repository. I would add that Git repository to Jenkins list of Global Shared Libraries. Then I would add a "@Library("my-support-library") _" line at the top of each Jenkinsfile. This would work - at least on paper. Is there any solution that allows me to share the logic if files are kept in Plastic repo(s)? When I Google on this most results say "use Shared Libraries". (Pushing out the build logic from Groovy to a different language - batch files, Python scripts, etc - will make it trivial to share the batch/Python/etc source files between the different build configurations, but then I lose access to all the existing Jenkins plugins and internal Jenkins state.)
  3. Jenkins build jobs often trigger twice

    Thanks for looking into this. I am running Jenkins 2.109, with PlasticSCM plugin 2.10. My test configuration is one machine that runs the Jenkins master (Ubuntu 16.04.4 LTS) with cm, plus one Jenkins build agent (Windows Server 2016) with cm (I will upgrade it to shortly.) The build jobs I run are Pipeline jobs that are set to "Pipeline script from SCM". The Jenkinsfile is written in Declarative Pipeline syntax. The Plastic workspace is ~4.5GB in size. Let me know if you need a smaller repro case, or want me to activate extra logging on our system, or anything else.
  4. Hi, we have used Jenkins for about half a year with Plastic SCM now. We have our main project set up to check Plastic for changes every 2 minutes. It detects when there are new changesets, but, after each such change it usually does an additional build even though there are no new changes in the Plastic repository. Sometimes it is zero, sometimes more than one, but one addtional is the typical count. Here is an example of what our build history looked like for a while today: All those builds labelled "Started by an SCM change" are caused by either a bug in Jenkins or a problem in the Plastic SCM plugin. Now when I have been reading up on it, I suspect the problem is within the Plastic SCM plugin. Here is a description of why a similar symptom for someone using Subversion is a problem specific to the Subversion plugin. Does anyone know of a solution? If not, is someone from Codice Software interested in debugging this together with me over the next few weeks? (If so, drop me a mail)
  5. Hi, I am about to move out some shared logic from our Jenkinsfiles to shared libraries, but I am running into problems. The Plastic SCM integration for Jenkins will always place content into a workspace folder beneath Jenkins' "base" project folder. This is different to how the Jenkins support for other SCMs (Git, Subversion, etc) works. This is fine for projects that are to be built. However, it seems that I cannot use Plastic SCM for shared libraries: Jenkins expects the src/ and vars/ folders to be directly in the "base" project folder, but with Plastic they will typically end up in the "base"/Jenkins-${JOB_NAME}-${NODE_NAME} folder. To be more concrete, on my Jenkins machine the shared library files will be checked out at /var/lib/jenkins/workspace/Tests/TestSharedLibrary@libs/FreedomJenkinsLibrary/Jenkins-TestSharedLibrary-MASTER but Jenkins requires the repository's files to exist at /var/lib/jenkins/workspace/Tests/TestSharedLibrary@libs/FreedomJenkinsLibrary The error from Jenkins is: ERROR: Library FreedomJenkinsLibrary expected to contain at least one of src or vars directories Do you have any solution for this? Otherwise, I may have to resort to putting the shared library code into GitHub, and that would make our operational processes more complicated. Corresponding JIRA in Jenkins CI: https://issues.jenkins-ci.org/browse/JENKINS-50284
  6. revInfo Owner is null before the cache

    Hi, I have sorted out the replication problems I had above (problem with network drivers and packet loss). However, I have a case which you may be interested in following up on for improving reliability in general: I use Plastic Cloud version I work against Plastic Cloud, with a local repository on my workstation. I had a consistent local repository on my workstation. Then, I performed a sync replication from Cloud to a local repository via the GUI, choosing "Pull visible". There were ~15 or so branches that were going to be pulled down. Partway through the replication I clicked "Cancel" in the sync replication progress window. After this, my local repository has been left in an inconsistent state. Attempting to update the workspace via the GUI resulted in "revInfo Owner is null before the cache" messages. Updating the the workspace via the commandline resulted in a number of "The revInfo.Owner is null before the cache" error messages intermingled between file download messages. A "checkdatabase" showed a number of messages such as "Seid 731 pointed by object 634548 does not exist in database rep_14.plastic.sqlite." When I checked with the GUI client it appears that I have replicated all the changesets from /main (but not all from a couple of child branches). I still got the errors when I attempt to switch to the newest changeset on /main. It looks to me like the replication operation within Plastic is not properly transaction-safe. Is this the case? If yes, is this something that you intend to improve upon in the future? I will make a backup copy of my sqlite databases; let me know if you need a copy of them for analysis. After this, I will drop my local "rep_14" repository database and re-replicate it from Cloud since this seems to be the only way for me to get back to a good state again.
  7. Related to your situation: We decided to train everyone on our team (progammers, designers, artists) to use the full Plastic SCM client. We are working against Plastic Cloud. We are missing out on the simplicity of Gluon, and we are not using exclusive checkout at all, but it has worked out well for us. Our process for a new developer is like this: 1. Begin by working directly against the main branch. Focus on the "Pending Changes" and "Branch Explorer" views. Do the first couple of check-ins in pairs until the developer is comfortable with checking in. Programmers typically work against the main branch for a day or so, and then they go straight to task branches. Artists and designers typically work against main for 2 weeks - 2 months. After that, they are ready for step 2. 2. Talk with the developer about how he/she are doing single, large check-ins because the developer is afraid of breaking things for others - and how the developer sometimes loses local work (typically through mistakes in the editor when making a large change, etc). Talk with the developer about the conflict ("check in often is good: backup & share results" vs "don't break the build"). Talk with the developer about how there currently is no way for him/her to share results between two people or two machines (say, work machine + home machine) without also sharing it with everyone else on the project. 3. Set the developer up with a personal branch. Talk with developer about how checking in and merging now are separate steps; checking in to a separate branch only has upsides, and no downsides; merging is where care needs to be taken. Do the first couple of merges in pairs until the developer is comfortable with merging. Explain how "merge from main to your branch often" makes merging back less complicated. Explain how the "merge down from main, then merge up" philosophy makes merging less complicated. Some of our artists choose to stop at stage 3. They want to work with personal branches. Those who want to move on, are ready for that after 3-6 months. 4. Talk with the developer about how their feature work is task oriented but how that structure is not reflected in the Branch Explorer - it's difficult to see where one effort ends and another ends in the project history. Explain the concept of "task branches". 5. Train developer in creating new branches for each major task. Small stuff can still go straight into the main branch. Discuss how to handle multiple overlapping tasks. Discuss how per-task branches help them organize their own work. We have all our programmers, 40% of our artists and 50% of our designers using per-task branches. Remaining developers have personal branches.
  8. The fix is live. I have confirmed it working in our Jenkins instance. Thanks!
  9. Thanks for the confirmation!
  10. ... I just realized: Perhaps Codice Software has fixed the bug in the Jenkins plugin source code, in an internal repository, but has not yet gotten around to publishing the bugfix to the rest of the world via https://github.com/jenkinsci/plasticscm-plugin ?
  11. Hi, we are successfully using Jenkins directly against Plastic Cloud (no GitServer intermediary involved). We are running into the same problem that @Misieq has reported: This bug is logged in Jenkins' issue database since some time back. Now it sounds as though this is something which Codice software is aware of, and has fixed, according to release notes for However, I have upgraded to Plastic SCM on my Jenkins master, and the problem remains. Same symptoms as in the Jenkins JIRA ticket. (I think it is odd that the latest Ubuntu package is version, when the latest readme on the Plastic site is for Not sure if that has any implication.) If someone from Codice software is interested in doing more troubleshooting, feel free to reach out to me.
  12. revInfo Owner is null before the cache

    Update on my situation: using Wireshark, I can see that a whole group of TCP packets (50 of them to be precise) are being dropped partway through a blob download from Azure. The random interruptions of "cm replicate" thus look like a problem with my home network. Apart from that; perhaps aggressive testing of randomly closing TCP connections mid-transfer during write operations to a Plastic repo would help Codice find some corner cases that can result in DB corruptions.
  13. revInfo Owner is null before the cache

    I guess you synchronized the repository where you had the initial failure and then you retry the replica using the command line, right? Yes. I also performed 'cm checkdatabase' -- this confirmed that the DB was not consistent. So I think there are two things, 1) I am having problems pulling and 2) the "cm replicate" command will, under some circumstances, result in inconsistent DB contents. At least when working against .sqlite format DBs. Even despite it using a transaction-based design (right?) The problems with pulling prevent me from "trying again": I have not yet succeeded in pulling down the repository to that computer in one go. I will do tests with a laptop in my home later; it will be connected over the same home Wifi. Will get back to you with info on how well that works. I have done a full "cm replicate" from the Cloud repo to a local repo on my office workstation now. That replication succeeded without any errors.
  14. revInfo Owner is null before the cache

    Hi, I am seeing similar symptoms on my home machine. I needed to pull down an entire new repository from Plastic Cloud to my home machine. Due to unknown reasons I had problems both downloading an updated Plastic Cloud installer. The download failed about 5 times throughout the 160MB download. This could either be my home network - the machine is connected to a home router over a wireless network - or problems with my ISP/Azure. I have not seen similar problems at our office. Anyhow, after installing a new version of the Plastic SCM client, I created a new sync view and attempted to pull a repository containing perhaps 250-500 branches and ~8GB of data. This failed within minutes. I resorted to doing "cm replicate /main/<child branch>@...@cloud <localrepo>@local" and fetching 1-2 weeks' of work at a time, as otherwise the command-line replication would also fail with messages like "The data for revision 236293 (segment 3) cannot be read from rep 8865.". I also interrupted a few replication operations with Ctrl-C. At one point, "cm replicate" complained that a changeset was locked. I stopped the local Plastic server and restarted it. The "cm replicate" operation was now capable of performing replications again. After I had replicated most of the branches, I went into the Plastic SCM GUI and performed a full sync via the sync view. No errors this time. After this, if I attempt to update the workspace (which previously was empty), after a few seconds I get a popup saying that the Plastic client failed updating certain files on my machine, with the reason "revInfo.Owner is null before the cache". Redoing the operation or forcing the operation does not help. I get messages like these in my server's plastic.debug.log.txt: 2017-11-30 16:44:47,829 W-39 00000000-0000-0000-0000-000000000000 kalms@falldamagestudio.com KALMSHOMEDESKTO DEBUG Transaction - Begin implicit transaction C:\Program Files\PlasticSCM5\server\rep_5.plastic.sqlite 2017-11-30 16:44:47,840 W-39 00000000-0000-0000-0000-000000000000 kalms@falldamagestudio.com KALMSHOMEDESKTO DEBUG BranchExplorerQuery - Read 286 branches 0ms 2017-11-30 16:44:47,871 W-39 00000000-0000-0000-0000-000000000000 kalms@falldamagestudio.com KALMSHOMEDESKTO DEBUG Security - SEID with id 494 was not found 2017-11-30 16:44:47,871 W-39 00000000-0000-0000-0000-000000000000 kalms@falldamagestudio.com KALMSHOMEDESKTO DEBUG Security - SEID with id 494 was not found 2017-11-30 16:44:47,871 W-39 00000000-0000-0000-0000-000000000000 kalms@falldamagestudio.com KALMSHOMEDESKTO DEBUG Security - SEID with id 494 was not found 2017-11-30 16:44:47,871 W-39 00000000-0000-0000-0000-000000000000 kalms@falldamagestudio.com KALMSHOMEDESKTO DEBUG Security - SEID with id 494 was not found I will make full logs available to you via email tomorrow. This does not block me from working. Actual questions to you: - Do you want more material for debugging? like, a snapshot of the SQLite database? (please be specific in which files in that case) - Is there a sure-fire way to get rid of this problem? other than, say, deleting the local replica and replicating it over again?
  15. Hi, this is mostly FYI. I have recently encountered the following error message during a pull operation: The sync process was unable to replicate the following branch: Branch: <branch name> Operation: Pull Source repository: <repo>@<organization>@cloud Destination repository: <repo>@local Error description: Explicit transaction expected, but found no transaction This happened earlier this week on a machine running Plastic SCM and just now on another machine using Plastic SCM The pull operation was pulling from Plastic Cloud to a server on my own workstation/laptop. I got this error message for two branches. Choosing "Retry" would successfully pull those branches. Several other branches were also included in the Pull operation; those pulled without any problems. Since then, I have performed more pull operations to pull newer changes from at least one of those branches, with no errors. (Suspicion: there was a small-scale, temporary hiccup with Plastic Cloud.)