Create and use a local mirrored yum repository for CentOS 6.4 x86_64 – Joe’s Blog

Create and use a local mirrored yum repository for CentOS 6.4 x86_64 – Joe’s Blog

A local mirrored yum repository is a copy of the standard yum repositories for CentOS that exists on your local network. It is kept up to date by synchronizing to an external site. If you have more than a few local hosts, using a local mirrored repo can significantly reduce internet traffic and speed up the update process.

This blog describes how to create, sync, test and use a local mirrored repo for CentOS 6.4. You will be shown how to create the repo, keep it in sync using cron, test the implementation and how to update the /etc/yum.repos.d directory on each host. It also provides 4 bash scripts that you can customize for your site to make maintenance easier.

Contents

  • Introduction
  • Site Configuration Parameters
  • Configuration and Synchronization Scripts

    • repo_conf.sh
    • repo_sync.sh
  • Initialize Your Mirrored Repository
  • Test Your Mirrored Repository

    • repo_test.sh
  • Cron the Synchronization
  • Use Your Mirrored Repository

    • repo_update.sh

Introduction

When you type “yum update” on a standard CentOS installation your site is updated from an external site that is randomly chosen from a list of mirrored repositories on the internet. The packages that need to be updated are detected by comparing version information from your site with the version information from the external site and then downloaded through the internet.

For a small number of hosts this works really well because most packages are small and there normally only a few that are updated at a time but once your network grows to more that a small number of hosts, keeping them all in sync can result in a fair amount of internet traffic.

Please note that the term “a small number of hosts” is deliberately vague. Some sites with a slow internet connection would probably see significant performance improvements once they reached 3 or 4 hosts whereas others with a very fast, fat pipe might not see any improvement until they had dozens of hosts. In my case I decided to switch over when I had about 20.

To reduce the internet traffic overhead associated with updating multiple hosts on your network, you can create a local mirrored repository that acts just like an external repository (which is, itself, mirrored). If you have a large site with a fast, fat internet pipe you should consider becoming an external repository to help the community.

This blog describes how to create the local repository, how to keep it synchronized, how to test it and how to update the /etc/yum.repos.d directories on your hosts to make it all work.

Site Configuration Parameters

The first thing that you need to do is determine where you want the local repository to reside. It must be a network resource that is visible to all of the hosts that want to use it. I use an NFS mount. For the purposes of this discussion we will assume that the network path to the mirrored repository is “/shared/repo/“.

Second you need to choose an administrative host that can create/update the repo directories and run the cron job to keep it in sync. For this discussion we will use the host “bigbob“.

Finally you need to determine which repositories you want to mirror. That decision is going to be site dependent but for me I chose the centos_6.4_extras, centos_6.4_os, centos_6.4_updates and elrepo_el6 repositories. The repositories with the “centos_” prefix. The repository with the “elrepo_el6” prefix is from the elrepo repository. It contains lots of useful stuff that is not in the standard release.

To see the repositories that you reference by default, take a look at the “/etc/yum.repos.d/*repo files on your system.

Here is a summary of the site configuration:

Network Path /shared/repo
Admin Host bigbob
Admin Path /shared/repo/bin
CentOS 5.4 Repos extras, os, updates
Other Repos elrepo
Conf Script /shared/repo/bin/repo_conf.sh
Sync Script /shared/repo/bin/repo_sync.sh
Test Script /shared/repo/bin/repo_test.sh
Update Script /shared/repo/bin/repo_update.sh

Configuration and Synchronization Scripts

To configure your site, log into bigbob (your administrative host) and make sure that you have the createrepo package installed. You can do this by running the “yum info createrepo” to see if it is installed. If it is, then you are good to go. If it isn’t then run “yum install -y createrepo“.

After that create the /shared/repo/bin directory so that you have a place to store the scripts and make sure the /shared/repo is available as an NFS mount (or automount). I am not going to describe how to create an NFS mount or automount for hosts on your site. It is a complex topic and there are many great resources available on the web.

repo_conf.sh

Now you need to create the site configuration file. This is a simple bash script that is used by the other scripts to get basic configuration information such as where the repo is located.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

#!/bin/bash

#

# Keep my local CentOS 6.4 repo mirror up to date. The local repo

# mirror is used by all CentOS 6.4 hosts on the internal nework.

#

mirror_repo=“/shared/repo”

if [ ! d $mirror_repo ] ; then

    echo “ERROR: mirror repo directory does not exist: $mirror_repo”

    exit 1

fi

#

# This is the generated repo file that is used in the

# /etc/yum.repos.d directory.

#

mirror_repo_file=“$mirror_repo/yum.repos.d/local.repo”

#

# This is the test directory.

#

mirror_repo_test=“$mirror_repo/test”

repo_sync.sh

Finally you need to create the synchronization script. This script also creates the mirror.

You will need to update the list variable at line 55 to change the external reference to an rsync site near you.

You will also need to update the excludes variable at line 66 to define which packages you wish to exclude. By default all packages are included.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

#!/bin/bash

#

# Keep the local mirror repo in sync with the external repo.

# This can be run in a cron jobs as follows:

#

#    # Update once per day at 10:00PM.

#    0 22 * * * /shared/repo/bin/repo_sync.sh

#

# The configuration information is /shared/repo/bin/repo_conf.sh.

#

umask 0

#

# Keep my local CentOS 6.4 repo mirror up to date. The local repo

# mirror is used by all CentOS 6.4 hosts on the internal nework.

#

medir=$(dirname